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Introduction 

After  devel  oping  the  initial  framework  in  firs  t  year  for  virtual  worlds  and  developing  the 
underlying  architecture,  this  year  our  experim  ent  focused  on  using  th  e  system  to  address  a 
clinical  area  .  We  focu  sed  on  card  iopulmonary  resuscitation.  Cardiop  ulmonary  arrest  (more 
commonly  known  as  cardiac  arrest)  is  the  absence  of  mechanical  activity  of  heart  or,  abrupt  loss 
of  functiona  lity  of  heart.  According  to  the  American  Heart  Association,  almost  80  percent  of 
cardiac  arre  sts,  which  occur  out  of  hospital  ,  are  witnessed  at  home  by  a  family  me  mber 
[33]. .Approximately  6.4  percent  of  the  patients  who  have  a  cardiac  arrest  ultimately  survive  [33], 
This  shows  the  importance  of  Adva  need  Cardiac  Life  Support  (ACLS)  skills,  which  requires  a 
team  to  perform  various  tasks  within  a  few  minutes  of  patients’  arrival  in  emergency  room.  It  is  a 
time-constrained,  sequ  ential  procedure  and  complex  team  work  that  requires  proper 
communication  and  coo  rdination  between  the  team  me  mbers  in  order  to  save  a  patient’s  life  . 
The  ACLS  t  earn  has  on  ly  five  minutes  to  perfo  rm  the  sequence  of  actions,  both  cognitive  (eg. 
decision-making  such  as  which  medicine  to  give,  diag  nosis  of  treatment  scenario)  and 
psychomotor  (eg.  CPR),  in  order  to  save  a  pati  ent.  It  all  depends  on  the  performa  nee  of  the 
team,  not  th  e  individuals,  during  the  procedure  that  they  f  ollow  to  save  the  patient.  If  a  team 
member  makes  a  mista  ke,  all  goo  d  work  don  e  by  the  te  am  will  become  insignif  icant,  and 
ultimately  the  patient  can  die. 

In  most  fields  in  which  time  is  the  most  important  factor  and  which  require  expertise  on  cognitive 
as  well  as  psychomotor  skills  for  better  decisi  on-making  skill  s,  novices  require  an  expert  t  o 
disseminate  knowledge  and  skills  to  them.  Theoretical  knowledge  can  be  learned  in  classroom 
environments  whereas  procedural  skills  and  communi  cation  skills  require  more  practice  to 
perfect.  This  approach  of  master-  apprenticeship  (or  app  renticeship  in  common)  model  of 
education  h  as  been  in  existence  for  a  long  ti  me,  where  master  performs  a  pro  cedure  and 
trainees  carefully  observe  the  procedures  and  practice  th  em.  From  t  heoretical  knowledge  to 
procedural  skills  to  co  mmunication  skills,  this  model  best  fits  the  requirements.  In  the  case  of 
psychomotor  and  communication  skills,  this  is  more  important,  because,  most  of  the  trainees  do 
not  have  an  y  idea  on  what  they  are  going  to  do  during  their  initial  lear  ning  phase.  However, 
there  is  a  limitation  that  at  any  point  of  time,  a  trainer  can  only  train  a  li  mited  number  of  trainees 
[1]. 

Hamman,  Beaubien,  and  Seiler  [34]  present  the  fact  that  errors  in  health  care  are  directly 
related  to  the  failures  in  the  structure  and  function  of  the  systems.  The  authors  also  mention  that 
team  training  is  given  less  prefere  nee  than  tr  aining  an  in  dividual,  although  most  of  the  care 
delivery  is  performed  by  teams  of  people.  As  m  entioned  earlier,  ACLS  is  a  team-based  time- 
critical  activity,  so,  in  order  to  deliver  better  care  to  the  patients,  we  need  to  understand  the 
importance  of  team  training  as  well  as  consid  er  more  eff  ective  ways  to  provide  training  fo  r 
teams. 

What  is  ACLS? 

ACLS  refers  to  certain  clinica  I  interventions  intended  to  treat  lif  e  threatening  medical 
emergencies  such  as  cardiac  arrest  and/or  respiratory  failure  etc.  To  master  ACLS,  it  requires 
extensive  medical  knowledge,  training  and  practice.  Only  qualified  healthcare  providers  such  as 
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physicians,  nurses  and  paramedics  etc.  can  provide  AC  LS  as  it  requires  several  qu  alified  skills 
such  as  understanding  emergency  pharmacology,  managing  patient’s  airway  and  interpreting 
electrocardiograms. 

Life  threatening  situations  as  cardiac  arrests  are  announced  as  a  code  blue  situation.  Code  Blue 
is  one  of  th  e  hospital  e  mergency  codes  that  ar  e  used  to  alert  several  emergency  situations  in 
the  hospitals  worldwide  so  that  the  staff  is  aware  and  able  to  react  t  o  the  situation  as  f  ast  as 
possible.  Code  Blue  means  that  there  is  a  patient  who  is  suffering  from  a  severe  life  threatening 
situation  and  he  requires  immediate  resuscitation  (needs  ACLS).  There  is  a  dedicated  team  of 
nurses  who  specialize  in  respondin  g  to  these  situation  s.  The  team  ne  eds  to  be  aware  of  th  e 
situation  quickly  and  act  accordingly  since  the  patient  needs  immediate  attention.  The  delays  in 
response  may  result  in  death  of  the  patient. 

Once  the  code  team  ha  s  assembled  they  have  to  follow  predefined  protocols.  The  y  have  very 
limited  time  to  react  an  d  it  is  literally  a  lif  e  or  death  situation.  They  are  bound  to  make  some 
mistakes  under  this  stre  ss  and  tension.  Our  h  ypothesis  is  that  we  can  decrease  the  error  rat  e 
significantly  if  we  focus  on  the  procedural  aspect  of  their  activities. 

ACLS:  current  training  approach 

Almost  all  patient-care  organizations  provide  regular  ACLS  training  to  facilitate  emergency  care 
providers  to  enhance  their  ACLS  skills.  In  a  typi  cal  trainin  g  session,  t  raining  team  members 
arrive  at  the  practice  room.  They  initiate  the  process  by  assigning  ro  les  at  first,  then  divide  the 
tasks  according  to  the  roles,  and  follow  the  ta  sks.  The  team’s  performance  is  monitored  an  d 
evaluated  by  an  e  valuator  throughout  the  period.  After  the  session,  the  evaluator  g  ives  a  final 
score  ba  sed  on  the  te  am’s  perfor  mance,  and  later  s/he  debriefs  wh  at  happene  d  and  what 
should  have  been  done  in  the  practice  room.  There  might  be  a  brief  did  actic  session  on  ACLS 
too.  After  th  e  debriefing  session  (an  d  the  did  actic  session),  the  team  will  perform  another  test  , 
and  the  team  is  e  xpected  to  perform  better  tha  n  the  previous  session.  The  same  evaluator  will 
evaluate  the  second  session  as  well. 

The  problem(s) 

Although  the  training  system  loo  ks  promisin  g,  there  are  various  issues  that  are  being 
compromised.  The  cost  associated  with  overall  setup  falls  on  a  higher  range,  and  the  time  taken 
for  training  takes  about  2  to  3  hours  to  complete.  Much  of  this  time  is  due  to  the  large  amount  of 
orientation  needed  for  training.  In  the  context  of  learning,  the  training  participants  are  not  guided 
during  the  practice  session.  So,  they  ha  ve  to  recall  what  they  had  le  arned  previ  ously  in  the 
didactic  session.  There  are  not  en  ough  trainers  to  provide  training  to  the  trainees  frequently 
because  of  which  trainees  get  less  time  to  practice  the  procedures  properly.  Apart  from  these, 
the  ACLS  training  se  ssions  occur  o  nee  in  every  three  weeks,  which  is  not  enough  for  practice 
when  we  consider  the  criticality  of  the  ACLS  skills. 

Learning  in  virtual  worlds 

With  rapid  development  of  comput  er  storage,  memory,  processors,  a  nd  high  speed  network 
infrastructure,  it  is  now  possible  to  create  a  virtual  reality  based  simulations  in  a  networked 
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(distributed)  environment  that  helps  users  to  learn  team  coordination  skills.  Computer  Supported 
Cooperative  Work  (CSCW),  in  gener  al  terms,  is  considered  to  be  a  colla  borative  work  done  by 
users  who  are  located  at  different  sites.  Tele  medicine,  tele-health,  t  ele-conferencing  all  are 
examples  of  CSCW.  When  CSCW  is  integrated  with  t  he  term  Virtual  Realit  y  (VR),  the 
environment  is  called  as  Collaborative  Virtual  Envi  ronments  (CVE),  or  simply  “Virtu  al  Worlds”, 
which  provide  immersive  virtual  environment  wh  ere  users  can  perform  various  actions,  and  can 
also  communicate  and  collaborate  with  others  in  the  envir  onment.  CVEs  have  been  used  in 
various  field  s  like  gaming  [4],  on  line  community  building  o  r  socia  lizing  [4,  5],  edu  cational  or 
working  environments  [6,  7],  CVEs  are  able  to  convey  th  e  social  dynamics  like  turn  taking, 
cooperation,  appraisal,  communication  to  users  in  a  proper  manner.  In  addition  to  that,  users 
can  be  assigned  different  roles  like  doctor,  patient,  trainer,  trainee  etc.  Current  CVEs  also 
support  diff  erent  media  required  for  communication  (t  ext,  audio,  video),  which  are  very 
important  for  group  discussions. 

How  virtual  worlds  can  persuade  users  to  change  their  behavior  and  attitude 

Because  of  the  features  that  virtual  worlds  provide,  they  h  ave  potential  to  chang  e  behavior 
and/or  attitude  at  different  situations  and  different  circumstances.  Fogg  mentions  that  there  are 
many  reasons  that  computers  can  be  better  persuaders  than  humans  [8].  Some  of  the  important 
reasons  are  :  computers  are  more  persistent;  t  hey  provide  greater  anonymit  y;  the  y  can  offe  r 
various  mo  dalities;  co  mputer  pro  grams  can  be  re-scale  d  as  per  u  sers’  need;  and  the  most 
important  one  -  “comp  uter  can  be  ubiquitous”.  Virtual  worlds  provide  all  these  fe  atures.  They 
are  more  persistent;  they  are  able  to  hide  users’  information;  various  input  output  methods  ca  n 
be  integrated  with  the  virtual  worlds;  and  can  be  modified  as  per  the  req  uirements.  In  presence 
of  internet,  virtual  worlds  can  be  accessed  from  any  part  of  the  world.  Hence,  we  can  say  that 
CVEs  are  a  n  integral  part  of  persuasive  frame  work  in  vari  ous  fields  like  gaming  (eg  World  of 
Warcraft),  communications  (eg.  virt  ual  shops:  Amazon.com,  eBay.com  etc.),  training  systems 
for  physical  exercise  (eg.  virtual  trainers:  Tri  pleBeat,  Wi  i  Fit  etc).  With  these  abilities, 
computerized  virtual  reality  based  interactiv  e  systems  have  potential  to  persuade  human  users 
in  the  field  of  education  as  well. 

Advantage(s)  of  training  in  virtual  worlds 

The  most  important  advantage  of  use  of  computer  based  simulation  in  the  field  o  f  education  is 
that  it  can  motivate  students  to  learn  and  practice  in  a  saf  e  environment  [9],  Simulation  also 
enables  students  to  practice  different  procedures  in  different  contexts  and  different  situations. 
Chodos  et.  Al.  suggest  that  virtual  world  simulations  consume  less  resources  and  are  capable  of 
providing  safe  and  realistic  e  nvironment  to  practice  [1],  The  added  persuasion  in  the  computer 
simulation  allows  students  to  learn  what  the  causes  are  and  the  effects  caused.  This  persuades 
students  to  enhance  their  skills  on  role-playing,  and  changing  their  attitudes  towards  different 
perspectives  [8]. 

Learning  in  virtual  worlds:  what  is  required? 

Research  on  team  training  in  CVEs  is  st  ill  at  its  infancy.  Current  appli  cations  of  C  VEs  do  not 
consider  implementation  of  time-critical  h  igh-performance  system.  They  still  la  ck  the  integration 
of  cognitive  task  a  s  well  as  psycho  motor  task  by  providing  an  intera  ctive  platform  to  users  to 
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perform  the  tasks.  There  are  many  team  based  activities  which  include  sequence  of  actions  and 
are  constrained  by  time  and  team-members  have  to  complete  their  task  within  the  specified  time 
frame  maintaining  h  igh  performance  level.  Re  search  studies  have  sh  own  adverse  effects  of 
time  criticality  on  performance  of  us  ers,  which  also  hinders  t  he  persuasiveness  in  the  CVEs.  In 
virtual  worlds,  it  is  more  likely  that  users  will  pay  less  attention  while  performing  time-critical 
procedure.  The  lack  of  verbal  interaction,  physical  cues  (like  facial  expressions,  eye  movement, 
movement  etc.),  and  psychological  cues  (like  feelings,  humor,  preferences  etc.)  ar  e  also  major 
barriers  for  the  implementation  of  persuasion  in  the  virtual  worlds. 

Contribution  and  hypothesis 

In  this  study,  we  try  to  a  ddress  the  issue  of  tea  m  training  in  time  critical  scenario  (A  CLS  in  our 
case)  and  also  the  learning  behavior  of  participants  in  different  scenarios:  when  the  participants 
are  provided  with  persuasive  elements  and  when  they  are  not.  We  then  discu  ss  whether  the 
participants  can  transfer  the  learned  skills  to  the  training  room  at  a  hospital.  Finally,  we  also  see 
whether  the  participant  s  can  retain  the  skills  in  the  virtual  world.  We  also  discuss  the  novel 
approach  of  integration  of  haptic  device  to  the  virtual  world  for  time  critical  activities  that  requires 
psychomotor  skills.  After  the  study,  we  predict  that: 

Hypothesis  1#:  Virtual  worlds  are  significantly  effective  in  delivering  team  training. 

Hypothesis  2#:  Participants  will  retain  the  skills  over  a  longer  period  of  time. 

Objectives 

CVEs  have  huge  pot  ential  to  pr  ovide  training  to  many  users  in  a  virtual  environment 
simultaneously.  Our  main  goal  of  this  study  is  to  design  and  develop  an  interactive  collaborative 
team  training  simulator  that  persua  des  users  t  o  perform  a  sequence  of  cognitive  as  well  as 
psychomotor  actions  in  time-constrained  environment. 

The  study  also  focuses  on  the  following  important  issues: 

•  Evaluate  the  validity  of  virtual  worlds  in  delivering  team  training  and  retention  over  a  long 
period  of  time. 

•  Monitor  and  record  a  ctivities  (and  hence  perfo  rmance)  of  users  while  performing  a 
collaborative  task. 

•  Create  an  online  result  sheet,  which  can  be  accessed  fr  om  anywhe  re  to  view  own 
performance.  (The  security  feature  of  the  perfo  rmance  sheet  can  be  customized:  teams 
can  view  only  their  results;  whereas  a  supervisor  can  view  all  results). 
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Background.  The  project  comme  need  in  October  2008.  From  a  fina  ncial  perspective,  many 
original  quotes  for  equipment  were  no  longer  valid  due  to  significant  price  increa  ses  of  the 
equipment  since  the  or  iginal  proposal  was  sub  mitted.  This  limited  the  ability  to  complete  th  e 
proposed  project  for  developing  physical  telemedicine  connections  across  the  western  region  of 
Banner.  Mo  re  importantly,  the  project  did  not  have  a  clinical  champion  as  th  e  Principle 
Investigator  and  that  would  have  been  a  m  ajor  roadblock  in  accomplishing  the  goals  o  f 
collaborative  telemedicine.  These  factors  were  recognized  within  the  first  three  months  of  the 
project,  and  at  which  stage  TATRC  was  informed  about  the  difficult  ies  that  had  arisen.  Arizon  a 
State  Unive  rsity  (ASU)  continued  to  develop  the  web  2.0  backbone  for  the  project,  but  t  he 
project  was  halted  at  t  hat  point.  A  t  this  stage,  we  contact  ed  TATRC  to  better  de  fine  a  new 
project  within  the  lines  of  military  relevance  and  of  importance  to  our  organization. 

Banner  Health  presente  d  a  new  pla  n  to  TATRC  and  it  was  approved  on  June  12,  2009.  The 
actual  proje  ct  started  in  July  2009.  Since  th  e  project  start  we  have  made  s  ome  rapid 
achievements  in  laying  the  foundation  of  the  virtual  world. 

To  lay  the  foundations  of  our  work,  we  will  present  the  related  work  and  then  highlight  our 
conceptual  framework 

2.  Related  Work 

We  sub  categorize  this  section  into  three  parts:  Team  Training,  Training  in  Virtual  Worlds,  and 
Persuasive  Technologies. 

2.1.  Team  Training 

Any  coordinated  effort,  performed  by  a  number  of  people  in  a  group  is  termed  as  team  work. 
Communication,  coordination,  cohesion  etc.  are  typical  characteristics  of  a  team.  All  the  team 
members  should  possess  these  skills  in  order  to  carry  out  assigned  task.  Team  trai  ning  is  very 
crucial  if  well  coordinated  team  work  is  required. 

Today,  almost  every  single  case  o  f  care  delivery  in  hospitals  or  outside  hospitals  involves  a 
team  of  healthcare  prof  essionals  yet  it  has  be  en  observed  that  more  often  than  not,  individual 
training  is  g  iven  more  importance  in  real  life  [27],  There  are  various  reasons  behind  this  fa  ct 
such  as  it  is  often  hard  to  set  up  training  se  ssions  a  ccording  to  each  individua  I’s  sche  dule, 
healthcare  professio  nal  trainees  are  from  disparate  lo  cations  etc.  These  healthcare  training 
programs  need  to  incre  ase  training  experience  of  working  in  interdisciplinary  teams  for  every 
individual  caregiver.  Hamman  W  et  al  demonstrated  tha  t  identifying  and  focusing  on  team 
critical  tasks  and  events  prior  to  and  during  the  training  respectively,  actually  lead  to  significant 
performance  improvement  in  teamwork  skills  [27]. 

Implicit  coo  rdination  is  one  of  t  he  charac  teristics  of  high  perfor  mance  teams,  where 
communication  overhead  is  very  less  because  t  he  participants  have  access  to  the  information 
without  asking  explicitly  [28],  Communica  tion  overhead  is  typically  the  cost  of 

communication/interaction  measured  in  time,  in  ternet  bandwidth  etc  [2  9],  Another  aspect  that 
vitally  affects  an  individual’s  ability  to  work  in  a  team  is  shared  me  ntal  models.  As  team 
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members  e  ngage  in  a  group  activity,  they  t  end  to  have  similar  th  oughts/ideas  in  order  to 
accomplish  the  task  which  ultimately  results  in  less  communication  across  the  team  [30],  These 
aspects  are  essentially  a  part  of  team  dynamics  which  is  important  to  be  considered  in  a  design 
phase  of  any  experimental  groupware  activity. 

Score  is  an  important  factor  in  motivating  participation.  Toups  Z  et  al  observed  that  if  points  are 
given  based  on  team  efforts,  participants  try  harder  to  work  as  a  team  and  accomplish  the  task 
in  a  well  coordinated  and  organized  team  effort  [31]. 

Advanced  Cardiac  Life  Support  (ACLS)  is  a  time-critica  I  activity  carried  out  by  a  dedicated  high 
performance  team.  Training  for  such  high  perfo  rmance  teams  is  in  real  life  scenarios  is  ne  ither 
possible  nor  advisable  since  it  is  generally  a  matter  of  someone’s  life.  Simulation  training  is  one 
of  the  best  solutions  available.  According  to  Wayne  et  al,  simulat  or  training  has  show  n 
significant  performance  improvement  in  a  team  of  physicians  while  performing  ACLS  [32]. 

2.2.  Training  in  Virtual  Worlds 

Based  on  their  purpose,  Collaborative  Virtual  Environments  (CVEs)  o  r  virtual  worlds  can  be 
categorized  into  one  of  the  followin  g  types:  ga  ming,  socializing  or  online  commun  ity  building , 
and  educational  or  working  environments  [19],  [19,  20]  outline  the  various  factors  that  need  to 
be  present  in  a  virtual  world  to  be  suitable  fo  r  education  al  purpose.  The  authors  compare 
various  CVEs  and  com  e  to  the  conclusion  that  selection  of  a  particular  CVE  depends  on  the 
purpose  of  the  training  system.  Below,  we  will  briefly  explain  the  resea  rch  on  CVEs  thatfocu  s 
on  healthcare  and  emergency  training. 

Wiecha  et  a  I  explored  the  potential  of  a  virtual  world,  Second  Life  (SL),  as  a  delive  ring  tool  for 
continuing  medical  education  (CME)  [10].  In  t  heir  study,  participants  had  to  sele  ct  and  a  djust 
insulin  level  for  patients  with  type  2  diabetes.  F  or  that  purpose,  participants  had  to  listen  to  an 
instructional  40-minute  insulin  therapy  talk.  Two  mock  patients  are  also  included  in  the  study  so 
that  the  participants  can  interact  with  the  patients,  and  discuss  within  th  emselves.  A 
questionnaire  was  provided  to  the  participants  before  and  after  the  t  alk  se  ssion.  The  stud  y 
shows  that  virtual  world  is  very  helpful  for  CME  education  by  showing  significant  increase  in  the 
score  after  the  talk  than  prior  to  it. 

Losh  [15]  lists  several  research  work  done  by  the  Interactive  Media  L  aboratory  at  Dartmouth 
Medical  School  based  on  virtual  environments.  Virtual  Clinic  is  one  of  such  work  where  a  virtual 
clinic  is  designed  by  following  the  master  floor  plan.  The  ma  in  objective  of  this  work  is  to  alio  w 
learners  lea  rn  about  social  behavior  and  vario  us  procedur  es  in  clinical  environments.  The 
Virtual  Terrorism  Response  Academy  (VRTA)  is  a  simulation  based  game  to  train  users  on  how 
to  act  during  crisis.  T  he  simulation  focuses  on  providi  ng  rescue  efforts  when  hazardous 
materials  are  involved.  Before  starting  the  game,  users  have  to  choose  and  assign  themselves 
a  ‘role’.  Based  on  the  role,  which  ca  n  be  a  fireman,  emerge  ncy  medical  technician  etc,  training 
is  provided  in  didactic  learning  space.  Quizzes  and  interactive  videos  are  also  included  in  order 
to  engage  the  users.  In  an  experimental  sessio  n,  a  scenario  is  provided  to  the  users  and  the 
main  objective  of  the  users  is  to  practice  with  radiation  meters  and  see  how  the  exposure  levels 
change  when  nearing  hazardous  objects. 
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Similar  to  VRTA  is  Play2Train  [18].  It  is  a  virtual  hospital  and  town  environment  which  is  created 
by  Idaho  Bi  oterrorism  Awareness  and  Preparedness  Program  (IDAPP).  The  rea  listic  virtual 
environment  of  Play2Train  provides  various  ki  nds  of  emergency  preparedness  videos  in  virtual 
classrooms,  and  also  supplements  several  tr  aining  exercises  to  pr  epare  users  in  case  of 
emergency  situations.  After  the  practice  session  s,  the  procedure  followed  by  the  students  can 
be  debriefed  by  the  instructor  to  clarify  the  experiences;  a  n  essential  part  of  simulation-based 
training. 

Callaghan  et  al  use  Second  Life  to  create  a  virtual  learning  environment  for  engineerin  g 
education.  They  demo  nstrate  various  intera  ctive  simulations  that  are  part  of  engineering 
education  [1 2].  Apart  fro  m  the  simulations,  a  virt  ual  lecture  theater  is  a  Iso  present  in  the  virtual 
world  which  contains  interactive  mini/main  lecture  slideshow  viewer,  media  centre  for  streaming 
video  content  and  message  center  s  for  feedba  ck.  As  Seco  nd  Life  doe  s  not  provid  e  SDK,  the 
authors  use  open  source  e-learning  software  SLOODLE  th  at  links  Second  Life  with  a  course 
management  tool  name  d  Moodle.  After  demonstration  of  the  simulations,  the  participants  ar  e 
asked  questions:  if  they  answer  it  incorrectly,  they  have  to  run  the  simulation  again  and  answer 
the  questio  ns  correctly.  However,  the  study  lacks  the  assessment  and  the  evaluation  of 
participants  and  they  mention  that  these  shortcomings  will  be  their  main  focus  in  the  future. 
Boulos,  Hetherington,  a  nd  Wheeler  [16]  describe  the  pote  ntial  use  o  f  Second  Life  in  medical 
and  health  education.  The  authors  provide  two  scenario  s  -  ‘Virtual  Neurological  Education 
Centre’  (VNEC.f http://www.vnec.co.uk)  and  ‘Healthlnfo  Island’  (http://infoisland.org/health  info). 
The  former  demonstrates  a  scenario  where  users  are  exposed  to  most  common  neurological 
disability  symptoms.  Apart  from  the  symptoms,  they  are  also  provided  with  related  information, 
events,  and  facilities  in  the  Secon  d  Life.  The  latter  involves  providing  training  p  rograms  for 
virtual  communities.  It  also  intends  to  provide  support  to  Second  Life  residents  by  pro  viding 
them  opportunities  to  participate  in  different  medical  groups  dealing  with  stroke  support, 
cerebral  palsy  etc. 

The  research  study  performed  by  Chodos  et  al  [1]  focuses  on  the  development  of  a  research- 
based  virtual  environment  to  enhance  communicati  on  skills  for  health  science  education.  They 
provide  two  case  studies.  The  first  one  is  the  development  of  EMT/ER  training  simulation,  which 
delivers  an  environment  to  train  EMT/ER  pers  onnel  on  ta  king  care  of  accident  victim  before 
taking  him  t  o  a  hospital.  This  case  also  focuses  on  excha  nge  of  patie  nt  information  between 
EMT  and  ER  personnel.  The  se  cond  case  is  designed  t  o  teach  various  compe  tencies  t  o 
students  like  rehabilitation  medicine,  nutrition,  physical  education  etc.  For  the  second  case,  the 
authors  design  a  simulation  in  order  to  increase  commu  nication  bet  ween  the  s  tudents  to 
develop  a  home-care  plan  for  e  Iderly  patient.  Based  on  the  case  st  udies,  they  discuss  the 
expectations  of  students  towards  virtual  world  based  learning  and  the  quality  of  learning. 

There  are  several  other  projects  th  at  focus  on  virtual  healthcare  system.  Second  Health  is  o  ne 
of  such  projects  where  users  can  learn  about  how  to  use  medical  devices  in  hospital  settings 
[12],  An  interactive  clin  ical  scenario  is  provided  to  learn  medical  device  training  in  simulated 
clinical  environment.  The  participants  are  provided  with  both  formative  and  summative  feedback 
during  the  training  session.  However,  the  system  does  not  provide  clinica  l-skills  train  ing 

component  in  a  collaborative  environment  wh  ere  multiple  users  mak  e  a  team  a  nd  perform  a 
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collaborative  task.  Similarly,  the  Ann  Myers  Medical  Centre  [13]  and  the  nursing  trainin  g 
program  from  Duke  University  [14]  provide  me  eting  places  for  medical  educators  and  students, 
where  instructors  can  present  lect  ures  and  present  educational  materials,  and  students  ca  n 
interact  with  each  other. 

2.3.  Persuasive  Technologies 

Various  researchers  have  worked  on  finding  appropriate  way  to  pers  uade  users  to  perform 
various  activities.  Fogg  [8]  defines  persuasive  technologies  as  “intera  ctive  computing  systems 
designed  to  change  p  eople’s  attitudes  and  b  ehaviors”.  He  list  s  various  persuasive  technology 
tools  (terminologies)  that  can  be  an  int  egral  part  of  any  system  in  order  to 
encourage/discourage  u  sers  to  per  form  some  actions  wit  hin  the  system  and  change  their 
attitude  and/or  behavior  while  doing  so.  In  medical  training/education,  persuasion  is  one  of  the 
most  important  factors  t  hat  can  affe  ct  the  perfor  mance  of  trainees/stud  ents.  Use  of  meaningful 
persuasive  components  (rewards,  realism,  social  presence  etc)  enhances  the  learning  where  as 
bad  design  of  persuasive  components  hinder  s  it.  In  th  is  section,  we  will  mention  some  of  the 
research  work  that  has  been  done  to  encourage  users  t  o  perform  activities  wit  hin  a  given 
system. 

Conradi  et.  al.  [17]  prop  ose  an  idea  of  collabor  ative  learning  through  p  roblem-based  learning 
(PBL)  in  Second  Life,  which  they  call  PREVI  EW.  Researchers  prep  ared  five  virtual  patien  t 
scenarios  for  learners,  which  were  later  delivered  to  the  le  arners  through  Second  Life  platform. 
The  main  objective  of  the  study  was  to  find  whether  computerized  simulation  based  PBL  can  be 
more  effective  than  cla  ssroom  based  PBL.  T  o  engage  students  effectively  in  training  the 
environment  provided  greater  realism,  active  decision  making,  an  d  suitab  le  collaboration 
environment  where  the  participants  can  interact  with  each  other.  The  study  shows  that  realism, 
and  suitable  interaction  environment  provided  b  y  Second  Li  fe  engages  students  eff  ectively  in 
learning. 

Consolvo  et  al  look  at  the  design  requirements  for  technology  to  enco  urage  physical  activity  in 
[21].  For  this  study,  th  ey  come  up  with  a  mobile  phone  applicat  ion  to  encoura  ge  users  to 
perform  physical  activity.  The  application  has  th  ree  different  versions:  b  aseline,  personal,  and 
sharing.  The  sharing  version  was  the  most  advanced  where  users  not  only  can  see  their  activity, 
but  also  ca  n  share  their  performan  ce  to  others  and  view  others  perfor  mance.  Based  on  their 
study,  they  describe  various  factor  s  that  motiva  te  users  t  o  perform  p  hysical  activity.  Giving 
proper  credit  on  comple  tion  of  ea  ch  task,  an  d  providing  personal  awa  reness  on  users’  past 
performance,  and  current  performance  are  the  basic  ele  ments  of  the  system  th  at  persuade  d 
users.  Another  important  factor  is  social  interaction.  According  to  the  authors,  social  influe  nee 
creates  social  pressure,  which  motivates  users  to  be  the  be  st  (or  at  lea  st  not  the  w  orst)  in  th  e 
society.  TripleBeat  [22]  is  also  a  similar  kind  of  mobile  phone  based  system  t  hat  motivate 
runners  to  achieve  predefined  exercise  goals  us  ing  musical  feedback  as  well  as  competition 
based  persuasion,  and  real-time  personal  awareness.  The  experiment  results  conclude  that  the 
system  is  “significantly  more  effective”  in  helping  runners  to  achieve  the  goals. 

How  blogs  and  podcasts  can  be  helpful  tools  to  provide  more  sense  of  community  in  a  group  is 
explained  by  Firpo  et  al  [23],  The  major  obje  ctive  of  their  study  is  t  o  change  attitude  and 
behavior  of  a  community  at  School  of  Informati  on  Systems  and  Technology  (SISAT)  in  order  to 
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foster  a  sen  se  of  community  amongst  its  memb  ers.  Based  on  the  funct  ional  triad  explained  in 
[8],  the  authors  conclude  that  social  presence  and  credibility  as  the  key  factors  to  persuade  the 
members  in  the  community. 

Several  virtual  reality  b  ased  games  have  already  evolved  to  motivate  users  to  ma  intain  good 
health.  The  following  simulation  based  applications  hav  e  proved  the  fact  tha  t  simulated 
environments  are  very  effective  to  change  one’s  attitude  and  behavior.  The  Tetrix  VR  Bike  [24] 
is  an  environmental  simulation  that  motivates  users  to  work  out  on  this  device  by  exploring  the 
virtual  environment.  The  faster  users  pedal,  the  faster  will  be  the  exploration.  Another  simulated 
environment  is  Bronkie  the  Bronchiasaurus  [25],  which  is  designed  to  help  kids  wit  h  asthma  to 
manage  their  condition.  The  study  s  howed  that  the  asthmatic  children  who  played  th  e  game  for 
at  least  30  minutes  report  incre  ased  self-ef  ficacy  to  take  care  of  their  chro  nic  condit  ion. 
Similarly,  HIV  Roulette  [26]  is  anot  her  simulati  on  to  provide  immediate  insigh  ts  into  sexua  I 
behavior.  Users  can  view  and  select  hypothet  ical  character  along  with  gender  and  behavior. 
Based  on  the  selectio  n  criteria,  the  system  rep  orts  whether  the  specified  behavior  is  likely  to 
cause  HIV  or  any  other  sexually  transmitted  diseases. 

3.  Conceptual  Design 

The  concept  design  of  the  system  is  shown  in  Figure  3.1.  The  main  objective  of  this  system  is  to 
allow  users  to  access  the  system  from  virtually  anywhere  (in  presence  of  internet  connectivity). 
The  figure  displays  real  world  and  virtual  world  sites.  Altho  ugh  the  team  members  are  in  th  e 
same  virtual  world  location,  they  actually  are  logged  in  from  different  locations  in  the  real  world. 
The  person  who  is  responsible  for  doing  CPR  has  access  to  the  haptic  device. 
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Figure  3. 1.  Concept  Design 
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3.1  Design  Components 

Collaborative  Virtual  Environment:  The  CVE  module  controls  all  t  he  visual  a  spects  of  th  e 
simulator.  We  choose  Active  World  as  our  CVE  in  order  to  design  and  implement  our  virtual 
hospital  in  the  world.  Users  can  log  in  to  the  Active  World  using  their  credentials.  After  logging 
into  the  system,  users  can  navigate  the  world  using  their  own  avatars.  Active  World  allows  up  to 
30  users  to  login  simultaneously,  which  provide  s  us  a  p  latform  to  perform  collabo  rative  work. 
The  users  can  interact  with  each  ot  her  through  text  messages  and  ca  n  also  see  each  other’s 
actions  within  the  virtua  I  environment.  Active  World  also  manages  the  visual  inst  ructions  that 
need  to  be  provided  to  an  “active  user”.  An  “active  user”  is  the  one  who  has  access  to  the  haptic 
device.  Apart  from  the  instructions,  feedback  on  the  user’s  performance  is  also  provided  through 
the  Active  World.  The  main  compo  nents  of  the  Active  World  that  we  use  for  this  r  esearch  are 
mentioned  below: 

•  Avatars: 

Avatars  are  the  representation  of  real  people  in  the  virtu  al  environment.  Active  world 
provides  customizable  avatar  list.  In  our  re  search,  we  use  our  own  avatar,  which  can 
perform  chest  compression  animation  sequence. 

•  Objects: 

In  the  Active  World,  most  of  the  things,  that  are  not  avatars,  are  ob  jects.  In  our  virtual 
world,  hospitals,  patients,  visual  cues,  instructions,  buttons  are  all  objects.  Different  kind 
of  action  sequences  can  be  applied  on  the  objects.  Act  ions  can  b  e  showing/hidin  g 
objects,  moving  objects,  etc. 

•  Animations: 

Animations  are  parts  o  f  action  se  quences.  A  part  from  showing/hidin  g  and  moving 
objects,  Active  World  can  also  perfo  rm  animations,  i.e.,  we  can  specify  separate  action 
sequences  at  the  beginning  and  the  end  of  the  animation  sequence. 

Haptic  module 

The  haptic  module  mai  nly  comprises  the  contr  oiling  of  ha  ptic  device  and  sendin  g  message  s 
from  the  haptic  module  to  the  AW.  The  haptic  device  was  integrated  to  the  AW  to  simulate  CPR 
action  in  the  CVE.  Novint  Falcon  is  preferred  to  be  used  in  our  study  because  it  is  cost  effective 
and  it  is  easier  to  setup  during  experimental  sessions.  When  a  participant  simulates  performing 
CPR  action  on  the  haptic  device,  it  t  riggers  an  event  in  the  AW.  The  event  that  gets  triggered  is 
the  “CPR  gesture”  in  the  AW  which  is  an  animation  sequence  of  avatar  performing  CPR  action. 
The  main  idea  of  using  the  haptic  device  is  to  give  the  perc  eption  of  performing  CPR  action  in 
real  world.  Haptic  rendering  in  this  module  is  the  simplest  one.  Spring  force  is  applied  when  the 
handle  oft  he  haptic  d  evice  is  pu  shed  towards  Y-axis.  As  the  force  resolution  of  the  Novint 

Falcon  haptic  device  is - only,  we  cannot  render  high  force  feedbacks.  This  would  affect  the 

performance  while  performing  CPR  as  the  surface  would  feel  soft.  To  simulate  harder  surface, 
we  place  a  real  spring  just  below  the  handle  of  the  device  so  as  to  render  harder  force  when  th  e 
handle  of  th  e  haptic  dev  ice  is  pushe  d  in.  As  we  are  concerned  only  with  the  calculat  ion  of  th  e 
distance  of  the  haptic  joystick  from  the  original  p  osition  to  the  end  position,  this  is  a  simple  but 
an  effective  way  to  increase  the  force  feedback  maintaining  consistency  of  the  haptic  loop. 
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Voice  module 

The  idea  of  integrating  voice  module  in  our  virtual  training  system  is  to  provide  mo  re  realistic 
team-work  and  team-coordination  while  performing  a  virtual  procedure.  Not  only  the  realism,  we 
also  needed  a  tool  for  making  calls  t  o  the  whole  team  (making  conferen  ce  call)  throughout  the 
procedure.  Based  on  the  popularity  and  cost-ef  fectiveness,  we  prefer  Skype  to  other  software 
that  provides  computer-to-computer  conference  calls  for  free. 

3.2  System  Design 

Figure  3.2  shows  the  system  design  that  integrates  different  components  of  the  syst  em.  It  also 
visualizes  the  information  flow  fro  m  one  module  to  others.  As  we  mention  earlier,  the  haptic 
feedback  is  local  only;  the  only  person  who  can  feel  th  e  force  fee  dback  is  th  e  one  who  is 
performing  CPR  on  ha  ptic  device.  When  the  participant  st  arts  performing  CPR,  it  triggers  the 
CPR  animation  sequence  in  the  AW,  which  is  visible  to  all  the  participant  s  who  are  logged  in  to 
the  system  at  that  moment.  Apart  from  animati  on  sequences,  the  syste  m  also  provides  visual 
cues  on  what  actions  the  participant(s)  have  to  do  next,  such  as  giving  medicine,  putting  oxygen 
bag  etc.  Ot  her  visual  f  eedback  co  mponents,  such  as  scores,  summary  sheets,  etc  are  also 
present  in  the  system.  Audio  components  are  also  made  available  in  order  to  provide  realism  to 
the  virtual  environment. 
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Figure  3.2.  System  Design 


The  users  can  make  voi  ce  calls  to  each  other  using  Skype  (free  computer-to-computer  calling). 
The  integration  of  Skype  with  the  system  has  two  major  benefits.  The  f  irst  one  is  that  it  provides 
realistic  tea  m  coordinat  ion  during  t  he  virtual  p  rocedure  w  here  they  h  ave  to  freq  uently  ask 
questions.  The  second  one  is  that  we  can  record  the  conversation  between  the  team  members 
and  evaluate  them  later  because  the  conversati  on  between  the  team  members  is  one  of  the 
major  factors  that  affect  the  team  performance. 
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4.  Methodology: 

One  of  the  several  required  skills  that  an  ACLS  team  member  should  possess  is  the  knowledge 
of  algorithms.  Each  te  am  me  mber’s  knowled  ge  should  be  evaluated  on  these  algorithms, 
decisions  regarding  the  patient  care,  leadership  skills  etc. 

Every  ACLS  effort  must  have  an  assembled  tea  m  and  a  de  signated  leader  who  assumes  the 
responsibility  of  overseeing  the  actions  of  the  team.  As  it  is  rightly  said,  “Too  many  cooks  spoil 
the  broth”,  if  there  is  more  than  on  e  person  making  decisions  regarding  patient  care,  it  ma  y 
result  in  tot  al  chaos.  T  he  team  leader  is  supp  osed  to  dire  ct  each  tea  m  member  and  oversee 
whether  they  are  following  his  com  mands  correctly.  In  resuscitatio  n  effort,  some  tasks  can  be 
managed  b  y  personnel  underwent  only  Basic  Life  Suppo  rt  and  others  require  an  advanced 
training. 

A  resuscitation  effort  typically  requires  coordination  of  four  critical  tasks: 

A  -  Maintain  the  Airway 
B  -  Breathing  for  the  patient 
C  -  Begin  compressions  -  if  no  pulse 
D  -  Defibrillate  if  required 

If  a  code  team  consists  of  5  individuals,  each  one  of  them  is  suppo  sed  to  assume  one  of  these 
critical  tasks  and  the  team  leader  oversees  their  actions.  If  the  team  members  haven’t  assumed 
these  roles  then  the  team  leader  should  quickly  assign  them  these  roles  as  they  assemble. 

Team  Leader 

Team  leader  has  important  responsibilities;  some  of  them  are  listed  below 

•  Assesses  the  patient. 

•  Ensures  that  each  team  member  is  performing  assigned  r  esponsibilities  correctly  and 
safely. 

•  Ensures  proper  interaction  and  communication  between  team  members. 

•  Ensures  that  defibrillation  occurs  on  time  with  adequate  safety. 

•  Ensures  safety  of  each  team  member  when  defibrillation  occurs. 

•  Decides  when  to  terminate  resuscitation  efforts  in  agreement  with  team  members. 

Airway  Management 

Some  responsibilities  of  a  person  who  is  designated  to  manage  airway  are  listed  below: 

•  Perform  the  head-tilt  /  chin-lift  action  so  that  the  patient’s  mouth  in  proper  position. 

•  Correctly  insert  an  oral  airway. 

•  Know  how  to  verify  placement  of  an  advanced  airway. 

•  Know  how  to  place  an  advanced  airway. 

Cardio  Pulmonary  Resuscitation 

Some  responsibilities  of  a  person  designated  to  perform  CPR  are  listed  below: 

•  The  ACLS  t  earn  me  mber  responsible  for  Cardio  Pulmonary  Resuscitation  must  know 
how  to  provide  compressions  of  adequate  rate  per  minute,  force,  depth  and  chest  recoil. 

•  The  person  performing  CPR  should  be  replaced  by  someone  else  on  the  team  every  two 
minutes. 

•  Patient  should  not  be  left  without  CPR  more  than  15  seconds  whenever  ACLS  is  in 
progress. 
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Electrocardiogram  Monitoring  and  Defibrillation 

Some  responsibilities  of  a  person  designated  to  perform  defibrillation  are  listed  below: 

•  Operation  of  Automated  External  Defibrillator  (AED)  and  manual  defibrillator. 

•  Accurate  placement  of  handheld  defibrillation  patches  and  hands  free  patches. 

•  The  safety  precautions  while  performing  defibrillation. 

•  How  to  troubleshoot  the  AEDs  or  manual  defibrillators. 

Vascular  Access  and  ACLS  Medication 

Some  responsibilities  of  a  person  designated  forgiving  medication  are  listed  as  below: 

•  How  to  perform  intravenous(IV)  access. 

•  Awareness  of  important  IV  fluids  of  choice  in  a  cardiac  arrest. 

•  Awareness  about  medications  to  be  given  according  t  o  the  ACLS  algorithm  and 
identified  rhythm. 

Resuscitation  basics 

While  the  code  is  in  pr  ogress,  the  team  leader  acts  as  th  e  single  point  of  authority.  He/she 
needs  to  take  initiative  and  assign  proper  roles  to  the  team  members.  The  leader  needs  to  guide 
the  team  through  various  resuscita  tion  algorithms.  As  a  team  leader,  h  el  she  needs  to  be  ope  n 
to  suggestions  and  ideas. 

During  a  re  suscitation  effort,  two  u  tmost  priorities  are  cardiopulmonary  resuscitation  and,  if  a 
shockable  rhythm  is  present,  defib  rillation.  Advanced  airway,  vascular  access  and  medication 
are  secondary.  The  next  steps  should  be  taken  according  to  the  rhythm  present  on  the  monitor. 
For  exampl  e,  if  the  monitor  does  n  ot  show  any  el  ectrical  activity  and  the  patient  is  in  cardia  c 
arrest  then  the  situation  is  identified  as  asystole.  If  the  monitor  shows  normal  sinus  rhythm  and 
there  is  no  central  pulse  present  then  it  is  ident  ified  as  pulseless  electrical  activity  (PEA).  If  the 
monitor  shows  VTach/VFib,  urgent  defibrillat  ion  is  highly  re  commended.  If  there  is  any  delay  in 
getting  defibrillator  ready,  CPR  s  hould  be  initiated  right  away.  If  t  he  patient  is  in  PEA  o  r 
Asystole,  defibrillation  is  not  recommended.  At  any  point  in  time,  while  the  ACLS  is  in  progress, 
if  patient’s  rhythm  changes  then  the  ACLS  procedure  should  be  changed  according  to  the  new 
algorithm. 

When  the  p  atient  is  ab  out  to  be  d  efibrillated,  everyone  s  hould  be  no  tified  to  be  clear  off  the 
patient.  Negligence  in  t  hese  activities  may  res  ult  in  severe  conseque  nces.  Once  the  shock  is 
delivered,  the  chest  compressions  and  oxygen  should  be  resumed.  Pulse  checks  should  take 
place  perio  dically  and  emergency  care  shoul  d  be  cont  inued  accord  ing  to  the  appropriate 
algorithm. 

Algorithms 

There  can  be  five  different  causes  of  pulseless  arrest  /  cardiac  arrest  namely, 

1.  Ventricular  Fibrillation 

2.  Ventricular  Tachycardia 

3.  Sinus  Bradycardia 

4.  Pulseless  Electrical  Activity  (PEA) 

5.  Asystole 

The  algorithms  has  a  main  fork,  the  rhythm  on  t  he  monitor  is  classified  as  shockable  /  not 
shockable.  We  have  chosen  two  algorithms  namely 
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1.  Ventricular  Fibrillation 

2.  Pulseless  Electrical  Activity  (PEA) 
from  each  of  these  two  classes. 

The  experiment  takes  place  in  the  Virtual  Wo  rid  where  provisions  ar  e  made  so  that  the 
participants  can  login  from  remote  locat  ions.  For  dat  a  colie  ctions  purpose  s,  all  the 
participants  will  be  present  at  one  central  site.  Their  goal  will  be  to  diagnose  a  patient  who  is 
having  a  cardiac  arrest  in  a  collaborative  effort.  There  are  two  modes  namely 

1.  Training  Mode 

2.  Testing  Mode 

In  the  training  mode  th  e  participan  ts  are  guid  ed  through  the  patient  treatment  scenario  by 
providing  instructions.  I  n  the  testin  g  mode,  participants  ar  e  supposed  to  perform  the  required 
actions  in  a  team  to  sa  ve  to  patient.  In  both  modes,  their  interaction,  different  gestures/actions, 
communication  etc.  is  monitored  and  recorded  in  the  database.  Participants  are  presented  with 
different  persuasive  elements  throughout  the  e  xperiment.  For  exampl  e,  They  are  given  point  s 
for  every  co  rrect  action  or  decision,  at  the  end  of  the  e  xperiment  they  are  prese  nted  with  a 
summary  screen  and  with  an  animated  character  of  a  physician  expressing  his  feedback.  If  they 
pass  the  test,  the  physician  is  happy.  If  they  fail  he  is  sad. 

Along  with  summary  screen,  they  are  given  a  lin  k  to  web  based  evaluation  sheet  wh  ich  details 
their  performance  on  the  test  /  experiment.  The  participa  nts  can  only  view  their  o  wn  evaluation 
sheets.  The  PI  has  access  to  a  II  evaluations/reports.  These  reports  will  be  anal  yzed  to  reach 
conclusions.  Below  are  some  screenshots  from  the  virtual  environment. 
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PULSELESS  ARREST 
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Continue  CPR  while  defibrillator  is  charging 
Give  1  shock 

•  Manually  biphasic:  device  specific 
(same  as  first  shock  or  higher  dose) 

Note  if  unknown,  use  200  J 

•  AED  device  specific 

•  Monop  hasic:  360  J 

Resume  CPR  immediately  after  the  shock 
Consider  antiarrhythmics;  give  during  CPR  (before 
or  after  the  shock) 

Amiodarone  (300  mg  IV/IO  once),  then  consider 
additional  150  mg  IV/IO  once)  or 
Lidocaine  (1  to  1.5  mg/kg  first,  then  0.5  to  0.75 
mg/kg  IV/IO,  maxium  3  doses  or  3  mg/kg) 

Consider  magnesium,  loading 

1  to  2  g  IV/IO  for  torsades  de  pointes 

After  5  cycles  of  CPR,  *  go  to  box  5  above 


During  CPR 

Rotate  compressors  every  2 

•  Push  hard  and  fast 

minutes  with  rhythm  checks 

(100/min) 

Search  for  and  b  eat  possible 

•  Ensure  full  chest  recoil 

conbibuting  factors: 

•  Minimize  interruptions  in 

Hypovolemia 

chest  compressions 

Hypoxia 

•  One  cycle  of  CPR:  30 

Hydrogen  ion  (acidosis) 

compressions  then  2 

Hypo/hyperkalemia 

breaths,  5  cycles  =  2 

Hypoglycemia 

minutes 

Hypothermia 

•  Avoid  hyperventilation 

Toxins 

•  Secure  airway  and  confirm 

Tamponade,  cardiac 

placement 

Tension  pneumothorax 

*  After  an  advanced  airway  is 

Thrombosis  (coronary  or 

placed,  rescuers  no  longer  deliver 

pulmonary) 

"cycles"  if  CPR.  Give  continous 
chest  compressions  without  pauses 
for  breaths  .  Give  8  to  20 
breaths/minutes.  Check  rhythm 
every  2  minutes 

• 

Trauma 

Figure  3.3.  Flow  chart  of  ACLS  procedure 
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Figure  3.4.  Beginning  of  an  experiment  (Training  Mode) 
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TURN  HAEPY  ANGRY  WAVE  JUMP  FIGHT  DANCE  BUN  CHKPULSE  CPR 


Start  CPR 
as  soon  as 
possible 


Figure  3.5.  User  is  instructed  to  start  CPR 
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Active  Worlds  -  Our  World  at  9S  20W  facing  SI 

Fie  Teleport  yew  Options  Show  j_ogn  Avatar  Vsbity  Tools  Web  Hefci 


j  DO©  0©6 


TURN  HAPPY  ANGRY  WAVE  JUMP  FIGHT  QANCE  EUN  CHKPULSE  CPR 


Click  on  the 
hands-free  patches 
to  place  them  on  the 
patient  and  try  to 
recognize  the  rhythm 
and  make  a  selection 


Oat 

y78jlo 


0.0  K/sec _ 33.6  frames/se  5:14:49  AM  Thu  Sep  30,  2010  &VRT  clock  200  meters _ 0.2  meters _ 0.0  m/s 
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Figure  3.6.  User  has  to  choose  the  probable  cause  of  the  patient  situation 
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Figure  3.7.  Patient  is  saved  when  the  user  performs  all  necessary  actions 
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5.  Persuasive  Framework 

Realistic  models,  environment,  and  presence  of  others  provide  better  persuasion  that  motivates 
users  to  perform  task  in  virtual  environments.  We  have  added  real  so  unds  and  noise  that  can 
be  heard  in  hospital  environment  in  real  world  (downloaded  from  website  link)  to  add  the  realism 
in  the  virtual  environme  nt.  We  have  also  inclu  ded  visual  effects  su  ch  as  information  boards, 
animation  sequences,  a  nd  pop-up  scores  which  will  al  low  users  to  know  what  they  have  to  do, 
what  they  are  doing,  and  how  th  ey  are  doing.  Presence  of  ot  hers  will  also  affect  the 
performance  of  individuals.  In  our  virtual  learning  environment,  more  than  5  users  are  logged  in 
simultaneously.  That  wil  I  make  user  s  feel  that  t  hey  are  under  surveillan  ce  in  the  vir  tual  worl  d 
and  that  will  eventually  motivate  users  to  perf  orm  to  their  best  during  procedures.  The  inclusion 
of  voice  module  will  also  provide  a  platform  to  communicate  with  each  other.  Parti  cipants  can 
ask  questio  ns  to  each  other  if  they  are  in  confusion.  Th  e  voice  mo  dule  enhances  the  tea  m 
coordination. 

The  different  types  of  persuasive  technology  tools  (defined  b  y  Fogg),  which  we  are  using  in  our 
system  is  explained  below: 

Tunneling 

It  is  a  process  in  which  users  are  “tunneled  through”  a  procedure.  Users  are  informed  to  perform 
task  1  (step  1)  and  the  n  guide  them  to  the  su  bsequent  steps  until  th  e  goal  is  obtained.  Our 
system  also  follows  this  mechanism.  When  the  participants  reach  the  virtual  ACLS  training  room 
and  start  doing  the  procedure  in  the  training  mode,  they  are  provided  with  information  regarding 
the  last  activity  as  well  as  what  are  they  supposed  to  do  n  ext.  Figure  3.5  shows  screenshot  of 
the  system  when  no  pulse  is  detected  and  the  board  is  placed  below  the  patient.  It  also  displays 
what  is  the  next  step  to  be  followed  (CPR  in  this  cas).  Likewise,  th  e  system  will  guide  the 
participants  until  they  obtain  their  goal,  i.e.  save  patient. 

Tailoring 

Tailoring  helps  in  two  ways:  first,  they  reduce  t  he  information  based  on  a  given  c  ontext  so  that 
the  information  wouldn’t  be  overwhelming  to  the  users;  second,  the  users  need  not  memorize  all 
the  steps,  they  can  easily  recall  what  they  know,  if  they  ar  e  shown  the  options.  In  our  system, 
there  are  various  checkpoints  where  users  have  to  decide  which  choice  would  be  the  best  one 
to  achieve  their  goal.  At  those  checkpoints,  we  provide  various  options  to  them.  They  will  make 
a  choice  and  follow  the  procedure  based  on  that  choice. 

Self-monitoring 

Another  important  criterion  for  persuasive  technology  is  to  a  Now  users  to  monitor  th  emselves.  It 
will  require  providing  them  formative  feedback  on  their  performance  during  a  procedure.  In  this 
system,  we  provide  scores  for  ea  ch  and  eve  ry  action  th  at  the  user  s  perform.  If  they  do  no  t 
perform  the  tasks  or  actions  in  a  correct  way,  or  within  the  pre-specifi  ed  timeframe,  then  they 
are  not  pr  ovided  any  points  on  those  a  ctions.  This  will  enable  users  t  o  monitor  their 
collaborative  work  at  any  instant  during  the  procedure. 

6.  User  experience: 

6.1.  Before  starting  the  training 

Participants  will  be  asked  to  fill  a  demographic  questionn  aire  before  the  experiment.  Every 
participant  will  be  give  n  a  unique  username  and  every  piece  of  information  will  be  kept 
confidential  and  will  not  be  disclo  sed  without  user  con  sent.  All  p  articipants  will  be  assigned 
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laptop  computers.  They  will  use  their  login  credent  ials  to  enter  the  virtual  environment.  Here 
onwards,  their  every  activity  will  be  monitored  and  recorded  in  a  secure  database. 

6.2.  During  the  session 

When  all  th  e  partici  pants  in  the  te  am  assemble,  the  expe  riment  will  begin.  Insid  e  the  virtual 
environment,  the  partici  pants  will  b  e  asked  to  navigate  around  the  virtual  hospita  I.  After  that, 
they  will  be  taken  to  the  emergenc  y  roo  m  whe  re  they  will  be  asked  to  diagnose  and  treat  a 
simulated  patient  who  is  having  a  cardiac  arrest.  They  a  re  expected  to  do  this  as  a  well 
coordinated  team  with  adequate  communicati  on.  The  participants  will  be  given  persuasive 
motivators  as  they  perform  the  exercise.  At  the  end  of  the  exercise,  they  will  be  presented  with  a 
feedback  summary  screen. 

6.3.  After  the  session 

They  will  be  given  acce  ss  to  the  ir  detailed  evaluation,  where  they  can  verify  their  performance 
on  the  experiment  and  evaluate  themselves  against  ideal  performance. 

7.  Conclusions 

With  the  system  completely  designed  we  are  now  ready  for  user  testing.  An  IRB  application  has 
been  drafted  and  is  ready  to  be  submitted  to  the  DoD  for  review. 

In  the  meanwhile  we  have  done  some  initial  testing  of  the  system.  Specifically  we  focused  on 
using  virtual  world  based  CPR  training  which  is  a  part  of  the  ACLS  training.  Kindly  see  Appendix 
I  for  the  experimental  results  which  show  the  validity  of  our  approach.  This  is  a  paper  ready  for 
review  and  to  be  submitted  to  Journal  of  Biomedical  Informatics. 

Another  important  work  in  our  efforts  has  been  to  use  Real  time  location  system  to  drive 
activities  in  Virtual  World.  This  work  has  been  accepted  for  publication  in  Journal  of  Biomedical 
Informatics.  See  Appendix  II  for  that  paper.  In  the  next  year,  we  will  link  this  system  with  our 
training  system  to  validate  if  the  training  does  produce  a  measurable  change  in  behavior  in 
trauma  and  critical  care  settings. 


22 


Key  Research  Accomplishments 


1 .  Development  of  virtual  world  which  is  based  on  floor  plans  of  training  hospital 

2.  Development  of  clinical  scenario  for  ACLS  in  the  virtual  world 

3.  Linking  haptic  devices  to  virtual  world  for  CPR  training. 

4.  Validation  of  the  CPR  training  module 

5.  Development  of  the  Persuasive  framework  in  virtual  worlds. 

6.  Design  of  experimental  framework  for  validation  of  the  virtual  worlds. 

7.  Cloud  based  reporting  system  for  evaluation. 

Reportable  Outcomes 

1.  Tutorial  by  Dr  Kanav  Kahol  at  A  merican  Medical  Inf  ormatics  Association  on  Virtual 
worlds. 

2.  Paper  titled  “Toward  Automated  Workflow  Analysis  an  d  Visualiza  tion  in  Clinical 
Environments,”  written  by  Mithra  Vankipuram,  Kanav  Kahol,  Trevor  Cohen  and  Vimla  L. 
Patel  accepted  for  publication  in  Journal  of  Biomedical  Informatics. 

3.  Paper  titled,  “Interactive  Haptic  Virtual  Collabor  ative  Training  Simulator  to  Retain  CPR 
Skills”  to  be  submitted  f  or  review  in  Ambient  Systems  conference  201  1  and  extended 
version  in  Journal  of  Biomedical  Informatics. 

Conclusions 

The  report  summarizes  some  of  the  key  efforts  made  in  virtual  world  based  tra  ining.  Th  e 

underlying  technology  has  been  developed  and  the  experi  mental  stage  is  about  to  begin.  The 
next  phase  of  human  testing  is  abo  ut  to  commence.  We  will  work  with  TATRC  to  ensure  goals 
are  met.  W  e  will  also  I  ay  the  foundation  of  evaluation  in  t  he  field  and  will  work  with  TATRC  to 
conduct  multi-institution  trials. 

We  also  plan  to  link  some  of  our  conventional  simulators  to  the  cloud  and  use  virtual  worlds  with 
simulation  tr  aining.  We  believe  that  would  be  a  major  advancement  of  the  science  of  virtual 
world.  Dr  Kahol  has  also  been  working  on  a  book  on  virtual  worlds  to  be  published  by 
Biohealthcare  Oxford  Publications  UK. 
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Abstract.  This  paper  provides  a  novel  approach  for  training  in  collaborative  environment  by 
integrating  collaborative  virtual  environment  (CVE)  and  haptic  joystick.  Active  World  is  used  as  the 
CVE  and  Novint  Falcon  is  the  preferred  haptic  device  to  send  force  back  to  the  user(s).  As  our  test 
scenario,  we  consider  CPR  skills  training  simulator  for  re-training  purpose.  CPR  is  not  just  a 
compress-and-release  procedure  -  it  is  a  collaborative  work  and  is  affected  by  the  performance  of 
each  team  member.  This  study  also  explains  the  transferability  of  the  CPR  skills  from  this  system  to 
the  real  world  case.  The  data  collected  12  participants  verily  that  this  simulator  provides  a  healthy 
improvement  on  the  performance  of  users  who  used  the  simulator  before  the  real  CPR  on  manikins 
compared  to  the  performance  of  the  users  who  didn’t  use  it. 

Keywords:  Collaborative  Virtual  Environment,  CVE,  Haptics  and  CVE,  Medical  education, 
Collaborative  Haptics  Training  Simulator,  CSCW  and  Haptics 

1  Introduction 

Computer  Supported  Cooperative  Work  (CSCW)  has  been  a  rapidly  growing  research  field  after  the 
invention  of  the  Internet.  According  to  Baecker  [1],  CSCW  is  an  activity,  which  is  performed  by 
groups  of  collaborating  individuals  in  a  computer-assisted  environment.  He  also  visualizes  CSCW  in 
a  form  of  2  X  2  matrix  of  location  (local  site,  different  sites)  and  time  (synchronous,  asynchronous). 
Most  of  the  time,  CSCW  is  considered  to  be  a  work  done  by  the  users  located  at  different  sites  [2], 

Collaborative  Virtual  Environment  (CVE)  is  generally  considered  as  the  combination  of  CSCW 
and  Virtual  Reality  (VR)  [6].  According  to  [2],  in  CVEs,  participants  share  a  common  virtual 
environment  and  are  connected  to  it  through  a  computer  network.  Participants  have  their  own 
avatar(s)  to  represent  their  identity,  location,  actions,  and  gestures.  Participants  are  also  able  to 
communicate  with  each  other  from  within  the  environment.  CVEs  are  best  suited  for  education  as 
they  are  capable  of  providing  group  discussion  (plain  text,  audio,  and/or  video),  and  can  also  support 
different  media  (text,  audio,  video)  to  display  information  about  particular  topics  to  the  participants. 
Dickey  [5]  mentions  that  a  CVE  consists  of  three  major  components:  (a)  3D  space  illusion;  (b)  a 
character  to  represent  real  user,  called  as  “avatar”;  and  (c)  an  interactive  communication  environment. 

Medical  education  is,  however,  slightly  different  from  traditional  fonn  of  education.  Apart  from 
cognitive  part  of  the  education,  developing  psycho-motor  skills  is  also  equally  important.  So  far,  most 
of  the  research  on  education  using  CVEs  is  based  on  disseminating  information  to  the  participants  in 
audio-visual  media  format.  There  have  been  numerous  virtual  reality  based  simulators  that  help  the 
participants  learn  psycho-motor  skills  in  addition  to  cognitive  skills.  However,  most  of  these 
simulators  are  stand  alone  and  only  one  participant  can  access  the  system.  In  real  world  emergency 
cases,  there  is  a  team  and  each  team  member  has  his/her  own  task  to  perfonn.  They  switch  their  role 
back  and  forth  during  the  same  emergency  care  session.  Clearly,  this  approach  of  educating  medical 
professionals  or  medical  students  is  not  suited  for  medical  education  because  not  only  are  the 
participants  communicating  with  each  other,  but  are  also  performing  psycho-motor  activities  at  the 
same  time. 

A  solution  for  this  can  be  a  networked  implementation  of  a  haptic  based  simulator,  where  each 
participant  can  perfonn  certain  task(s)  in  a  local  system  and  send  the  haptic  and  visual  information  to 
a  remote  system  [7].  However,  network  delay  affects  the  perfonnance  of  the  system  as  it  is  a  very 
tedious  work  to  maintain  high  haptic  rate  in  presence  of  network  delays.  This  leads  to  inconsistent 
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haptic  feedback  and  results  in  system  failure  most  of  the  time.  To  ensure  stable  network  based  haptic 
systems,  several  network  based  algorithms  to  control  the  delay  in  the  network  must  be  implemented. 
That  would  be  a  separate  research  topic  in  itself. 

In  this  paper,  we  attempt  to  solve  the  issue  of  collaborative  medical  education  by  providing  a  novel 
approach  of  integrating  CVE  with  haptic  joystick.  We  focus  on  the  chest  compression  part  of  the  CPR 
to  re-train  the  users  who  already  know  how  to  perform  CPR  but  haven’t  practiced  it  for  some  time.  A 
participant,  who  has  access  to  the  haptic  device,  must  maintain  the  rate  of  100  compressions  per 
minute  while  performing  CPR  on  the  haptic  device.  The  participant  is  provided  with  haptic  feedback 
at  real  time.  Based  on  his/her  performance  s/he  is  given  visual  feedback  in  the  CVE,  so  that  s/he  can 
improve  his/her  performance.  His/her  performance  can  also  be  seen  by  other  users  who  are  logged 
into  the  system  at  the  same  time.  This  study  also  intends  to  check  whether  the  participants  retain  the 
CPR  skills  afterwards. 

In  traditional  CSCW  matrix  (by  Johansen  and  Baecker),  most  of  the  CSCW  systems  were  based  on 
visual  mode  (texts  and  graphics).  For  newer  generation  of  CSCW  systems,  it  is  difficult  to  visualize  a 
system  that  has  a  part  running  on  local  system  and  another  part  running  in  local  as  well  as  remote 
systems.  Hence,  we  also  add  a  new  dimension  to  the  CSCW  matrix  -  “display”,  to  properly  visualize 
the  type  of  CSCW  system.  By  “display”,  we  refer  to  the  output  modality  of  the  system.  The  new 
CSCW  3D  matrix  is  more  intuitive  to  know  about  the  system  that  is  being  represented.  The  idea  of 
CSCW  3D  matrix  is  picturized  in  Figure  (1). 

The  paper  is  organized  as  follows:  we  outline  related  work  in  Section  [Related  Work],  The  overall 
system  design,  methodology,  and  implementation  of  the  simulator  is  explained  in  Section  [System 
Design ].  The  experimental  design,  setup,  and  participants  are  described  in  Section  [Experimental 
Design ].  The  results  obtain  from  the  simulator  are  presented  in  Section  [Results  and  Discussion] . 
Finally,  Section  [Conclusion  &  Future  Work]  concludes  the  paper. 

2  Related  Work 

Several  research  groups  have  developed  various  CVEs  in  the  past.  Based  on  their  purpose,  they  can  be 
categorized  into  one  of  the  following  types:  gaming,  socializing  or  online  community  building,  and 
educational  or  working  environments  [12].  However,  only  a  few  selected  CVEs  can  be  used  in  the 
field  of  collaborative  education.  [12],  [13]  outline  the  functionalities  that  should  be  present  in  a  CVE 
in  order  to  be  suitable  for  educational  purpose.  The  authors  take  into  consideration  the  systems 
Croquet  (www.opencroquet.org),  Worlds  (www.worlds.net),  Active  Worlds 
(www.activeworlds.com),  Second  Life  (www.secondlife.com).  There  (www.there.com),  Tixeo 
(www.tixeo.com),  I-maginer  (www.i-maginer.fr),  and  Moove  (www.moove.com)  as  test  CVEs  for 
collaborative  educational  virtual  environment  (CEVE).  Each  of  these  CEVEs  is  tested  for  the 
predefined  functionalities  in  order  to  choose  virtual  learning  platfonn.  Although  the  authors  choose 
Croquet  in  [12]  and  Second  Life  in  [13],  they  also  mention  that  selecting  a  particular  CVE  depends  on 
the  purpose  of  the  training  system. 

Boulos,  Hetherington,  and  Wheeler  [3]  describe  the  potential  use  of  Second  Life  in  medical  and 
health  education.  They  provide  two  scenarios  -  ‘Healthlnfo  Island’  (http://infoisland.org/health  info) 
and  ‘Virtual  Neurological  Education  Centre’  (VNEC,  http://www.vnec.co.uk).  The  former  involves 
providing  training  programs  for  virtual  communities.  It  also  intends  to  provide  support  to  Second  Life 
residents  by  providing  them  opportunities  to  participate  in  different  medical  groups  dealing  with 
stroke  support,  cerebral  palsy  etc.  The  latter  demonstrates  a  CVE  where  users  are  exposed  to  most 
common  neurological  disability  symptoms.  Apart  from  the  symptoms,  they  are  also  provided  with 
related  information,  events,  and  facilities  in  the  Second  Life.  Conradi  et.  al.  [4]  propose  an  idea  of 
collaborative  learning  through  problem-based  learning  in  Second  Life.  Researchers  prepared  five 
virtual  patient  scenarios  for  learners,  which  were  later  delivered  to  the  learners  through  Second  Life 
platform.  They  also  conclude  that  the  CVE  engages  students  effectively  in  training  by  providing 
greater  realism,  active  decision  making,  and  suitable  collaboration  environment. 

The  evolution  of  haptic  devices  has  provided  immense  opportunities  for  researchers  to  develop 
haptics  based  virtual  medical  training  simulators.  The  research,  in  return,  could  create  a  new  way  of 
training  medical  students  and/or  practitioners.  Cheaper  haptic  devices  are  available  in  the  market 
these  days,  which  makes  virtual  medical  training  even  simpler  and  easier.  Research  projects  like  [8] 
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for  needle  based  procedure,  [9]  for  open  surgery,  [11]  for  minimally  invasive  surgery  are  only  a  few 
of  those  haptic  based  medical  training  simulators  that  gave  medical  education  a  new  dimension. 

Pascale,  Mulatto,  and  Prattichizzo  [10]  proposed  and  implemented  a  new  idea  of  using  haptic 
device  in  a  CVE.  The  idea  behind  the  research  was  to  help  blind  people  to  navigate  around  Second 
Life  with  the  help  of  joystick.  They  used  Phanom  3DOF  haptic  device  for  haptic  rendering.  Whenever 
the  user  encountered  an  obstacle,  the  system  renders  some  force  to  the  user  through  the  haptic  device. 
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Fig.  1.  a)  Conventional  CSCW  matrix;  b)  new  CSCW  3D  matrix. 


3  System  Design 

This  section  is  divided  into  three  sub-sections:  Design  Components,  Methodology,  and 
Implementation. 
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3.1  Design  Components 

Collaborative  Virtual  Environment  (CVE):  The  CVE  module  controls  all  the  visual  aspects  of  the 
simulator.  We  choose  Active  World  as  our  CVE  in  order  to  design  and  implement  our  virtual  hospital 
in  the  world.  Users  can  log  in  to  the  Active  World  using  their  credentials.  After  logging  into  the 
system,  users  can  navigate  the  world  using  their  own  avatars.  Active  World  allows  upto  30  users  to 
login  simultaneously,  which  provides  us  a  platform  to  perfonn  collaborative  work.  The  users  can 
interact  with  each  other  through  text  messages  and  can  also  see  each  other’s  actions  within  the  virtual 
environment.  Active  World  also  manages  the  visual  instructions  that  need  to  be  provided  to  an  “active 
user”.  An  “active  user”  is  the  one  who  has  access  to  the  haptic  device.  Apart  from  the  instructions, 
feedback  on  the  user’s  perfonnance  is  also  provided  through  the  Active  World.  The  main  components 
of  the  Active  World  that  we  use  for  this  research  are  mentioned  below: 

•  Avatars: 

Avatars  are  the  representation  of  real  people  in  the  virtual  environment.  Active  world 
provides  customizable  avatar  list.  In  our  research,  we  use  our  own  avatar,  which  can  perform 
chest  compression  animation  sequence. 

•  Objects: 

In  the  Active  World,  most  of  the  things,  that  are  not  avatars,  are  objects.  In  our  virtual  world, 
hospitals,  patients,  visual  cues,  instructions,  buttons  are  all  objects.  Different  kind  of  action 
sequences  can  be  applied  on  the  objects.  Actions  can  be  showing/hiding  objects,  moving 
objects,  etc. 

•  Animations: 

Animations  are  parts  of  action  sequences.  Apart  from  showing/hiding  and  moving  objects, 
Active  World  can  also  perform  animations,  i.e.,  we  can  specify  separate  action  sequences  at 
the  beginning  and  the  end  of  the  animation  sequence. 

Haptic  module:  The  haptic  module  mainly  comprises  the  controlling  of  haptic  device  and  sending 
messages  from  the  haptic  module  to  the  Active  World.  A  user,  “active  user”,  has  access  to  the  haptic 
device,  in  the  real  world,  and  can  perform  CPR  on  the  material  mounted  on  top  of  the  haptic  device. 
While  performing  compression  during  CPR,  the  user  gets  an  appropriate  amount  of  force  feedback. 
Based  on  his  performance,  a  message  is  sent  to  the  Active  World,  and  the  system  provides  feedback  in 
the  fonn  of  visual  cues.  The  purpose  of  the  feedback  is  to  help  the  user  to  speed  up  or  slow  down  the 
rate  of  compression  and  recoil. 

Figure  (2)  shows  the  overall  design  of  the  system.  The  direction  of  the  line  indicates  the 
information  flow. 


Fig.  2.  System  Design 
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3.2  Methodology 

This  research  is  part  of  a  mock  code  simulator  in  which  people  work  in  a  collaborative  environment. 
The  major  objective  of  this  system  is  to  re-train  users  to  perfonn  CPR  skills,  who  already  know  how 
to  do  it,  but  haven’t  practiced  for  some  time.  Each  user’s  performance  is  displayed  in  the  CVE  (active 
world).  To  achieve  our  goal,  we  divide  the  objectives  into  a  set  of  subtasks.  Figure  (3)  shows  the 
sequence  of  tasks  followed  while  designing  the  simulator. 

In  the  figure,  the  first  module  calculates  force.  When  participants  perfonn  compression,  a  force  is 
calculated  and  is  applied  in  the  opposite  direction.  The  handle  of  the  haptic  joystick  moves  back  to  the 
original  position,  thus  simulating  ‘recoiling’.  Mass-spring  model  is  used  for  its  simplicity  and 
efficiency  in  our  system  for  force  calculation.  In  the  proposed  system,  the  force  feedback  is 
independent  of  the  visual  feedback  -  it  is  not  necessary  to  detect  collision  between  the  objects  in  the 
CVE.  This  is  very  important  for  a  collaborative  work  like  CPR  because  not  all  members  in  a  team 
perform  the  same  task  simultaneously;  however,  the  members  switch  roles  every  now  and  then. 

After  calculating  the  force,  the  number  and  the  rate  of  compressions  is  calculated.  The  following 
algorithm  is  used  to  count  the  number  of  compressions. 

Initialize  startPosition  at  the  beginning  of  the  haptic  loop, 

Initialize  countCompression  =0;  and  touchMin,  touchMax  to  false, 


Fig.  3.  Tasks  flow  of  the  system 

Obtain  currentPosition,  and  calculate  distance  =  currentPosition  -  startPosition, 
if  distance  <=  0,  set  touchMin  =  true, 
if  distance  >  1.5  (inch),  set  touchMax  =  true, 
if  both  touchMin  and  touchMax  are  true,  then 

if  distance  <=  0,  then  //when  it  goes  back  to  the  original  position 
increment  countCompression, 
set  touchMin  =  false, 
set  touchMax  =  false, 

Repeat. 

Where,  touchMin  and  touchMax  represent  the  planes  when  the  handle  of  the  haptic  joystick  is  at 
the  initial  position  and  compressed  position  respectively.  The  number  of  compressions  and  the  rate  of 
compressions  are  then  stored  in  the  local  computer,  which  can  be  used  for  the  validation  of  the 
proposed  system. 

Once  the  number  and  the  rate  of  compressions  are  calculated,  callback  functions  (aw  object  add, 
aw  object  change)  are  used,  event  handlers  and  attributes  ( A  W  O  B  J  E C'T  D  E S  C  RIPTI  ON , 
AW  OBJECT  ACTIN)  provided  by  Active  World  SDK  are  used  to  visualize  the  data  in  the  CVE. 
These  data  can  be  seen  by  all  the  users  who  are  available  at  the  virtual  training  location.  The  output(s) 
of  the  system  is  shown  in  Figure  (4). 


3.3  Implementation 

The  proposed  system  follows  a  distributed  architecture.  The  CVE  acts  as  a  server  to  disseminate 
information  to  all  users,  and  runs  at  a  remote  site.  As  discussed  earlier,  there  are  many  CVEs 
available;  however,  Active  Worlds  is  best  suited  for  our  requirements.  Active  World  provides  its  own 
SDK  that  helps  to  communicate  and  interact  with  the  Active  World  system  from  native  C/C++ 
language.  Messages  can  be  sent  from  the  local  computer  to  the  Active  World  server  following  the 
protocol  mentioned  in  the  SDK.  The  SDK  also  helps  to  add,  remove,  and/or  modify  objects  in  the 
world  in  the  real  time. 

For  the  haptic  rendering  part,  we  used  Novint  Falcon  haptic.  Novint  Inc.  provides  TIDAL  haptic 
API  to  communicate  with  the  Falcon  haptic  device.  The  API  provides  different  control  mechanisms  to 
control  the  position  of  the  device  and  the  force  being  rendered  to  the  user.  The  TIDAL  API  is  available 
in  C/C++. 


a)  Green  board  for  correct  compression-rate. 


b)  Red  board  for  lower  compression-rate. 
Fig.  4.  Screenshots  from  the  system 


4  Experimental  Design 

Each  participant  has  to  perform  3  CPR  trials.  In  the  first  trial,  the  participants  have  to  perform  CPR 
without  any  feedback.  They  have  to  maintain  the  rate  of  100  compressions  per  minute.  The  second 
trial  provides  visual  cues  and  feedback.  Participants  have  to  synchronize  their  rhythm  of  compression 
with  the  visual  cues  provided  on  the  screen.  A  ‘Press’  button  is  used  as  a  visual  cue  and  it  becomes 
visible  and  invisible  maintaining  the  rate  of  100  per  minute.  Participants  have  to  perfonn  compression 
whenever  the  button  appears  on  the  screen,  and  recoil  when  it  disappears.  In  addition  to  the  visual 
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instructions,  the  participants  are  given  feedback  on  their  performance.  They  are  shown  their 
compression-rate,  number  of  compressions  and  a  message  (if  needed).  If  their  compression-rate  is  less 
than  90,  then  the  message  “Go  Faster!”  is  displayed,  and  the  participants  must  increase  the  rate  of 
compressions.  Similarly,  when  the  compression-rate  is  more  than  110  “Go  Slower!”  message  is 
displayed.  These  two  messages  are  shown  in  red  background,  representing  that  they  are  deviant  to  the 
actual  rate.  If  their  performance  is  between  97  and  103  compressions  per  minute,  the  current 
compression-rate  and  the  number  of  compressions  is  shown  on  a  green  background.  The  third  trial  is 
similar  to  the  first  one;  no  visual  cues  and  feedback  are  provided. 

For  each  trial,  the  number  of  compressions,  time  taken  for  each  compressions  (in  seconds),  and  the 
rate  of  compression  is  stored.  Our  hypothesis  is  that  participants  perform  better  when  they  are 
provided  feedback  (second  trial)  than  when  they  are  not  (first  trial). 

Experiment  Setup:  12  participants  (3  female,  9  male)  participated  in  the  experiment.  All  participants 
had  basic  idea  about  CPR  skills  and  had  already  perfonned  CPR  before.  All  of  them  already  knew 
that  they  had  to  maintain  the  rate  of  100  compressions  per  minute.  However,  they  haven’t  perfonned 
it  for  2  months.  Only  5  participants  had  experience  on  using  the  haptic  device  before  the  study,  all 
others  were  using  the  haptic  device  for  the  first  time. 

Before  the  trials,  each  participant  was  explained  about  the  system,  the  experimental  design,  and 
what  they  had  to  do  in  the  experiment.  Each  participant  was  provided  a  minute  to  get  used  to  with  the 
simulator.  When  they  were  ready,  the  first  trial  was  perfonned.  An  interval  of  approximately  0.5 
minute  separated  the  trials.  Figure  (6)  shows  how  the  system  was  set  up  for  the  experiment. 

5  Results  and  Discussion 

Figure  (6)  displays  the  number  of  compressions  in  each  trial  performed  by  each  participant.  The  safe 
range  (90-100  compressions  per  minute)  is  highlighted  in  the  figure.  For  each  trial,  performance 
metrics  of  each  participant,  like  number  of  compression,  time,  and  rate  of  compression,  were 
recorded.  Almost  60%  of  the  participants  could  not  maintain  their  rate  within  the  range  of  90- 1 1 0 
compressions  per  minute.  The  outcome  shows  that  people  who  know  about  CPR,  but  do  not  practice  it 


Fig.  5.  Experiment  Setup:  CVE  shown  at  the  left  and  haptic  device  at  the  right 

often,  tend  to  make  mistake  in  maintaining  the  required  compression-rate.  The  compression-rate 
varied  from  76  to  126  per  minute.The  second  trial  was  performed  in 

presence  of  visual  cues  and  performance  feedback.  All  participants  were  able  to  maintain  the 
compression-rate  between  90  and  1 10.  The  range  of  number  of  compressions  per  minute  varied  from 
95  to  104  in  the  second  trial.  Participants  performed  better  in  the  third  trial  as  compared  to  the  first 
one.  All  of  them  were  able  to  maintain  the  compression  between  90  and  100.  The  compression-rate 
varied  from  90  to  110. 


Participants 

Fig.  6.  Performance  of  12  participants  on  each  trial  (safe-range  is  highlighted). 

We,  initially,  hypothesized  that  the  participants  should  maintain  the  rate  between  90  and  110,  and 
providing  them  the  visual  cues  and  feedback  would  improve  their  performance.  In  the  visual  cues, 
we  displayed  their  compression-rate,  and  whether  they  are  going  fast  or  slow,  in  real  time,  so  that 
even  if  they  were  maintaining  the  required  rate,  they  can  improve  to  maintain  it  at  1 00  compressions 
per  minute.  Our  hypothesis  was  verified;  when  they  were  provided  the  cues,  they  performed  much 
better  than  when  they  were  not  given  feedback  on  their  performance.  The  compression-rate  varied 
from  95  to  104,  which  was  more  than  that  we  expected. 

In  the  third  trial,  we  wanted  to  check  whether  the  participants  could  retain  the  skill  or  not  if  visual 
cues  are  not  provided.  From  the  outcome  of  the  experiment,  we  can  say  that  they  could  retain  the 
skills.  All  participants  could  maintain  within  the  range  90-1 10. 

The  important  thing  to  mention  here  is  that  the  participants  did  not  practice  on  this  simulator  for  a 
long  period  of  time.  Based  on  the  results,  even  in  that  short  period  of  time  they  spent  with  the 
simulator,  they  could  retain  the  skills.  It  is  possible  only  because  they  already  knew  how  to  perform 
CPR.  This  proves  that  this  simulator  helps  the  people,  who  already  know  about  CPR  skills,  to  perform 
better  by  providing  feedback  on  their  performance  and  to  retain  the  skills. 

5  Conclusion  and  future  work 

This  paper  focused  mainly  on  interactive  collaborative  CPR  skills  training  simulator  for  the  purpose 
of  re-training  the  users,  who  already  know  how  to  perform  CPR  but  had  not  practiced  for  some  time. 
To  achieve  our  main  objective,  we  also  presented  a  novel  approach  of  integrating  haptics  and  CVE 
by  localizing  haptic  feedback  and  transfering  the  positions  of  the  device  to  the  server  located  at 
remote  site.  In  the  experiment,  the  participants  performed  CPR  at  local  site.  We  performed  three 
trials:  first  trial  without  any  feedback;  second  trial  with  feedback;  third  trial  without  feedback.  The 
participants  performed  a  lot  better  in  the  second  and  the  third  trials  than  compared  to  their 
performance  in  the  first  trial.  Other  users,  who  were  logged  into  the  virtual  world  during  the  trials, 
could  also  see  the  performance  of  each  participant. 

This  opens  up  different  possibilities  that  we  can  do  with  the  integration  of  haptics  and  CVE.  Apart 
from  re-training  skilled  CPR  practitiners,  this  system  can  be  a  part  of  a  virtual  mock  code  training 
simulator  [Sai’s  paper]  where  participants  perform  different  tasks  during  emergency,  and  they  also 
need  to  switch  tasks  in-between.  The  participants  can  log  in  from  different  locations,  and  can 
interactively  practice  the  mock  code  in  the  virtual  environment. 

This  can  also  be  used  as  virtual  assessment  tool  for  CPR  skills.  Participants  can  log  in  from  a 
remote  site  to  perform  CPR.  His/her  performance  will  be  stored  in  a  server,  where  only  authorized 
user  can  login  and  evaluate  the  performance. 


Installing  haptic  devices  at  different  locations  and  performing  a  collaborative  tasks  in  CVE  will  be 
the  future  work  that  we  will  be  focusing  on.  The  future  work  also  includes  identify  proper  depth 
compressed  during  CPR. 
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Abstract 


Lapses  in  patient  safety  have  been  linked  to  unexpected  perturbations  in  clinical 
workflow.  The  effectiveness  of  workflow  analysis  becomes  critical  to  understanding  the 
impact  of  these  perturbations  on  patient  outcome.  The  typical  methods  used  for 
workflow  analysis,  such  as  ethnographic  observations  and  interviewing,  are  limited  in 
their  ability  to  capture  activities  from  different  perspectives  simultaneously.  This  limita¬ 
tion,  coupled  with  the  complexity  and  dynamic  nature  of  clinical  environments  makes 
understanding  the  nuances  of  clinical  workflow  difficult.  The  methods  proposed  in  this 
research  aim  to  provide  a  quantitative  means  of  capturing  and  analyzing  workflow.  The 
approach  taken  utilizes  recordings  of  motion  and  location  of  clinical  teams  that  are  ga¬ 
thered  using  radio  identification  tags  and  observations.  This  data  is  used  to  model  activ¬ 
ities  in  critical  care  environments.  The  detected  activities  can  then  be  replayed  in  3D 
virtual  reality  environments  for  further  analysis  and  training.  Using  this  approach,  the 
proposed  system  augments  existing  methods  of  workflow  analysis,  allowing  for  capture 
of  workflow  in  complex  and  dynamic  environments.  The  system  was  tested  with  a  set 
of  15  simulated  clinical  activities  that  when  combined  represent  workflow  in  Trauma 
units.  A  mean  recognition  rate  of  87.5%  was  obtained  in  automatically  recognizing  the 
activities. 

Key  Words —  clinical  workflow;  complex  systems;  medical  errors;  radio  identification; 
virtual  reality;  visualization 
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1  Introduction 

1.1  Workflow  and  medical  error 

The  health  care  industry  faces  a  number  of  challenges  and  arguably  one  of  the  most  im¬ 
portant  ones  lies  in  maintaining  high  levels  of  patient  safety.  A  much-cited  report  re¬ 
leased  by  the  Institute  of  Medicine  (IOM)  estimates  that  as  many  as  98,000  people  die 
each  year  due  to  medical  errors.1  Medical  errors  cause  more  deaths  annually  when 
compared  to  motor  vehicle  accidents,  breast  cancer  or  even  HIV.  Consequently,  scien¬ 
tists  across  the  world  have  actively  researched  the  nature  of  medical  errors  and  the  ac¬ 
tivities  that  cause  the  errors.  An  integral  component  of  this  research  effort  is  workflow 
analysis. 

A  workflow  is  a  description  of  a  sequence  of  operations  or  activities  performed 
by  various  entities  or  agents  in  the  system.  It  provides  a  description  of  the  context  and 
conditions  in  which  errors  occur.  Careful  analysis  of  workflow  can  be  employed  to 
model  the  distribution  of  cognitive  work  and  the  information  flow  in  complex  environ- 
ments.  For  example,  Malhotra  et  al.  utilized  ethnographic  observations  and  interview 
data  to  analyze  the  workflow  in  an  intensive  care  unit.  The  workflow  served  in  the  de¬ 
velopment  of  a  cognitive  model  from  which  details  of  infonnation  flow  could  be  ex¬ 
tracted.  This  type  of  analysis  could  lead  to  the  discovery  of  latent  systemic  flaws  that 
potentially  result  in  adverse  events.  In  addition,  monitoring  and  assessment  of 
workflow  in  complex  clinical  environments  can  provide  clues  on  the  efficacy  of  patient 
management.  For  these  reasons,  workflows  in  clinical  environments  are  an  important 
aspect  of  patient  care  and  safety  research. 

Recent  research  has  approached  the  study  of  social  systems  such  as  clinical  en¬ 
vironments,  using  scientific  theory  based  on  complex  adaptive  systems .4  A  complex 
adaptive  system  is  defined  as  a  dynamic  network  of  entities  acting  simultaneously 
while  continuously  reacting  to  each  other’s  actions.5'6  The  control  in  such  systems  is 
often  decentralized  and  highly  dispersed.  The  overall  behavior  is  the  result  of  a  large 
number  of  decisions  made  at  every  instant  by  individual  agents  or  entities  in  the  envi¬ 
ronment.  In  such  environments,  the  inherent  complexity  of  interactions  makes  captur¬ 
ing  and  analyzing  workflow  difficult  and  often  unpredictable.  These  difficulties  can  be 
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attributed  to  a  disassociation  between  the  complex  nature  of  the  environment  and  the 
tools  available  to  analyze  the  workflow  in  such  environments. 

Critical  care  environments  are  complex  and  dynamic  and  change  constantly 
with  stress  levels.  The  changes  in  staff  continually  alter  team  dynamics  and  the  excess 
of  technology  and  equipment  creates  unforeseen  demands  on  clinicians.  These  charac¬ 
teristics  allow  for  the  categorization  of  critical  care  environments  as  complex  adaptive 
systems.  Consequently,  the  problems  with  workflow  analysis  in  complex  adaptive  sys¬ 
tems  are  applicable  to  critical  care  environments  as  well. 

The  tools  currently  used  for  workflow  analysis  in  clinical  environments  include 
methods  such  as  ethnographic  observation,  shadowing  of  individual  clinicians,  surveys 
and  questionnaires.7  These  are  qualitative  tools  of  analysis.  The  data  collected  by  these 
methods  can  be  used  to  model  segments  of  the  workflow  centered  on  a  particular  indi- 
vidual  and  their  activities.  While  the  workflow  generated  in  this  manner  captures  many 
aspects  of  the  overall  workflow,  more  often  than  not,  certain  key  pieces  of  information 
may  never  be  captured.  For  example,  observations  are  gathered  from  an  individual’s 
point  of  view  and  may  not  be  adequate  to  capture  multiple  activities  occurring  at  an  in¬ 
stance  of  time.  Theoretically,  by  increasing  the  number  of  observers  in  the  clinical  envi¬ 
ronment  it  is  possible  to  capture  information  about  the  activities  in  the  environment 
from  several  perspectives.  However,  based  on  informal  interviews  conducted  with 
Trauma  clinicians,  more  than  two  observers  are  often  considered  to  be  disruptive  to  the 
clinical  workflow.  With  such  constraints  imposed  on  data  collection  in  complex  envi¬ 
ronments,  there  is  a  need  for  an  unobtrusive  alternative  that  can  augment  existing  me¬ 
thods  of  data  collection  and  enable  piecing  together  a  more  complete  workflow,  from 
individual  and  team  perspectives. 

1.2  Black-box  approach  to  capturing  clinical  workflow 

An  example  of  complex  social  system  that  is  similar  to  a  clinical  environment  is  avia¬ 
tion.  A  critical  component  of  error  analysis  in  aviation  is  the  black  box.  The  black  box, 
as  a  tangible  unit  refers  to  devices  installed  on  aircrafts  that  track  both  communication 
within  the  cockpit  of  the  aircraft  as  well  as  performance  parameters  such  as  altitude, 
airspeed  and  heading.  From  a  conceptual  perspective,  the  black  box  is  a  constant  moni- 
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toring  tool  that  does  not  interfere  with  the  procedures  of  aviation  and  simply  monitors 
parameters  pertaining  to  the  flight. 

In  clinical  environments,  tools  that  are  conceptually  similar  to  a  black  box 
would  be  able  to  monitor  workflow  continuously.  In  addition,  the  tools  would  monitor 
the  workflow  without  disrupting  the  activities  of  entities  in  the  environment.  In  this  pa¬ 
per,  we  propose  a  system  that  utilizes  the  black-box  approach  for  continuous  workflow 
monitoring  and  analysis  in  clinical  environments.  This  system  employs  radio  identifica¬ 
tion  technology  (RID)  to  offer  continuous  monitoring.  RID-enabled  tags  are  portable 
electronic  devices  used  to  uniquely  identify  entities  in  an  environment.  These  tags  can 
be  used  to  continuously  monitor  agents  and  artifacts  in  the  clinical  environment.  Basic 
information  about  interaction  between  the  entities,  such  as  duration  of  proximity  and 
location  can  be  recorded.  This  data,  combined  with  qualitative  measures  provide  an  in¬ 
termediate  workflow  that  can  be  visualized  in  three-dimensional  (3D),  virtual  reality 
(VR)  environments.  We  can  combine  this  system  with  conventional  means  of  ethno¬ 
graphic  observations  for  high-resolution  capture  of  workflow.  The  end  result  is  a  sys¬ 
tem  that  enables  the  capture  and  visualization  of  workflow  in  complex  environments 
for  the  purpose  of  enhanced  workflow  analysis. 

2  Background  and  Related  Work 

Methods  used  to  analyze  workflow  in  clinical  environments  can  be  one  for  two  types  - 
qualitative  methods  or  quantitative  methods.  While  qualitative  methods  involve  subjec¬ 
tive  observations  gathered  by  researchers,  quantitative  methods  typically  involve  the 
usage  of  sensor  technology  or  video  recordings  to  capture  data  about  workflow.  The 
main  differences  between  the  data  captured  using  quantitative  methods  and  qualitative 
methods  are  as  follows: 

(i)  Using  quantitative  methods,  accurate  time-stamped  data  can  be  obtained.  Hu¬ 
man-intensive  methods  can  only  produce  near-accurate  time-stamped  observa¬ 
tions. 
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(ii)  Qualitative  methods  of  data  collection  produce  relatively  low  volume,  high 
quality  data.  On  the  other  hand,  quantitative  methods  produce  a  high  volume  of 
abstract  data  that  in  some  way  indicates  underlying  workflow. 

(iii)  Human-intensive  qualitative  methods  best  suited  for  low-intensity  situations 
while  automated  quantitative  methods  are  optimal  for  data  gathering  in  high- 
intensity  situations. 

The  following  sections  detail  the  existing  qualitative  and  quantitative  methods 
used  to  analyze  workflow  in  critical  care  environments. 

2.1  Qualitative  methods  for  workflow  analysis 

Malhotra  and  his  colleagues  analyzed  the  workflow  in  the  intensive  care  unit  (ICU)  in 
order  to  understand  the  process  of  evolution  of  error  in  a  critical  care  setting.  Ethno¬ 
graphic  observations  and  interviews  are  utilized  to  gather  data  to  model  workflow  cen¬ 
tered  on  the  entities  and  activities  in  the  environment.  The  process  of  gathering  obser¬ 
vations  involves  following  a  key  member  of  the  critical  care  team  and  recording  all  of 
their  interactions  with  other  members  of  the  team,  patients  and  equipment.  These  key 
players  are  then  interviewed  to  corroborate  the  observations  collected  and  to  delineate 
their  individual  workflows.  Using  observations  and  interviews,  a  collective  workflow  is 
reconstructed  by  combining  the  individual  workflows  of  each  key  player.  The  devel¬ 
oped  workflow  summarizes  how  ICUs  function  and  where  errors  are  most  likely  to  oc¬ 
cur. 

Laxmisan  and  her  colleagues  utilize  ethnographic  observations  and  interviews 
to  analyze  the  workflow  in  an  emergency  department  (ED)  as  well.  The  workflow  is 
analyzed  to  study  the  cognitive  demands  imposed  by  the  workflow  in  the  context  of  the 
work  environment.  Multi-tasking,  interruptions,  gaps  in  information  flow  and  handov¬ 
ers  during  shift  change  are  some  of  the  aspects  of  the  workflow  that  are  studied  in  de¬ 
tail. 

2.2  Quantitative  methods  for  workflow  analysis 

Quantitative  methods  provide  some  means  of  gathering  infonnation  about  the  activities 
and  whereabouts  of  entities  in  an  environment.  An  entity  could  be  a  person  (nurse,  phy¬ 
sician,  patient  etc.)  or  a  machine  (such  as  ultrasound  machine).  The  tracked  activities 
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can  then  be  pieced  together  (similar  to  integration  of  observations  and  interviews)  to 
provide  a  complete  picture  of  the  workflow. 

The  sensors  typically  used  for  entity  activity  recognition  include  passive  infra¬ 
red  sensors,  radio  identification  tags  and  pressure  sensors.  The  sensors,  depending  on 
their  type  are  utilized  to  detect  various  activities  that  the  entity  is  involved  in.  A  number 
of  systems  have  been  developed  for  activity  recognition  and  workflow  monitoring  us¬ 
ing  different  types  of  sensors.  These  systems  use  the  various  types  of  sensors  in  some 
combination  to  model  key  activities  of  the  entity  being  tracked.  In  general,  these  sen¬ 
sors  are  encased  into  a  physical  form  representing  a  tag.  These  tags  can  sense  different 
types  of  information  like  movement  and  location  through  the  ensemble  of  sensors  em¬ 
bedded  in  the  physical  fonn. 

In  the  domain  of  healthcare,  tags  have  been  employed  for  tracking  patients, 
equipment  and  staff  to  improve  patient  care  and  the  efficiency  of  clinical  workflow9'15. 
Fry  and  Lenert15  developed  a  system  for  location  tracking  of  patients,  staff  and  equip¬ 
ment  called  MASCAL.  The  main  aim  of  the  system  was  to  aid  in  streamlining  patient 
care  during  mass  casualty  situations.  RID  tags  are  used  by  the  system  to  track  the  loca¬ 
tion  of  key  players  (clinicians  and  equipment)  in  patient  care  during  emergencies.  This 
information  is  integrated  with  personnel  databases,  medical  information  systems  and 
other  applications  (such  as  those  that  enable  registration  and  triage)  in  order  to  central¬ 
ize  the  management  of  resources  during  critical  situations.  In  addition,  MASCAL  in¬ 
cludes  interfaces  for  centralized  management  of  various  entities  in  the  system. 

Chen  et  al.14,  studied  the  incorporation  of  RID  technology  in  a  clinical  setting  in 
non-psychiatric  hospitals  in  Taipei,  Taiwan.  Tags  were  used  to  identify  patients,  and 
notify  clinicians  on  the  status  of  patients  and  patient  related  information  (lab  reports, 
radiology  results  etc.).  Preliminary  studies  showed  that  using  the  RID  enabled  frame¬ 
work,  decreased  the  length  of  waiting  for  patients  in  intensive  care  units. 

The  other  technique  for  activity  monitoring  is  processing  of  video  recordings. 
Hauptmann  et  al.16  describe  a  system  that  recognizes  activities  from  videos  captured 
using  video  processing  techniques.  The  system  was  developed  to  recognize  activities  of 
daily  living  (ADL)  for  patients.  Examples  of  ADL  activities  include,  visiting  the  wa- 
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shroom,  eating,  sleeping  etc.  Cameras  placed  at  key  locations  within  the  environment 
provide  video  feeds.  These  video  feeds  are  processed  to  identify  the  patients  and  hence 
draw  conclusions  on  the  possible  activities  the  patients  were  involved  in. 

2.3  Limitations  of  qualitative  methods 

Qualitative  methods  are  human-intensive,  i.e.  they  require  significant  amounts  of  hu¬ 
man  effort  for  data  gathering  and  analysis.  The  dependency  of  qualitative  methods  on 
human  effort  has  certain  advantages  and  disadvantages.  The  main  advantage  is  that 
human-intensive  methods  usually  yield  data  that  are  of  high  quality.  These  data  are  de¬ 
tailed  and  descriptive,  and  potentially  insightful  inferences  can  be  made  using  qualita¬ 
tive  analyses  of  these  descriptions.  The  disadvantage  however,  is  that  the  dependence 
on  people  for  data  gathering  and  analysis  limits  the  capabilities  of  these  methods  to 
capture  important  details  of  the  collective  workflow  in  a  critical  care  environment. 

Observation  gathering  is  a  classical  qualitative  method  for  workflow  analysis 
that  suffers  from  its  dependence  on  human  effort.  It  is  difficult  for  individuals  to  moni¬ 
tor  and  document  all  activities  that  occur  at  every  instant  in  a  dynamic  and  complex  en¬ 
vironment.  Interviews  on  the  other  hand  suffer  from  the  poor  recall  of  events  on  the  part 
of  clinicians  being  interviewed.  Facts  about  events  may  be  altered  as  the  memory  of  the 
event  changes  temporally  (post-hoc  bias).  Other  real-time  methods  of  data  collection 
such  as  audio  and  video  recording  systems  not  only  require  consent  from  clinicians  to 
be  used  to  gather  workflow  data,  but  also  require  significant  human  efforts  for 
processing  data  collected  to  retrieve  meaningful  information.  Post-processing  of  real¬ 
time  data  involves  manual  analysis  of  audio  and  video  data  in  order  to  detect  various 
workflow  events.  The  real-time  data  is  then  manually  annotated  with  the  key  events  that 
have  been  detected.  This  process  requires  time,  effort  and  sufficient  researcher  expertise 
in  order  to  be  completed  successfully.  Such  limitations  make  these  methods  more  suited 
for  workflow  analysis  in  simple,  low-activity  environments. 

2.4  Limitations  of  quantitative  methods 

In  most  quantitative  methods,  sensors  for  monitoring  activities  and  locations  are  placed 
at  pre-defined  locations.  The  rigid  infrastructure  often  makes  installation  costs  prohibi¬ 
tive.  In  addition,  maintenance  can  be  complicated  if  spatial  configurations  are  altered. 


Another  issue  lies  with  the  modeling  approaches  employed  to  track  workflow.  In  all  the 
current  systems,  the  sensor  system  is  employed  to  determine  the  location  of  the  entities 
from  which  activities  are  estimated.  This  system  works  well  if  the  location  identifica¬ 
tion  is  reasonably  accurate.  However,  RID  systems  can  often  be  highly  erroneous,  re- 
suiting  in  close  to  200%  errors  in  location  estimates  .  Location  in  these  systems  is  de¬ 
termined  by  geometric  triangulation  methods  that  have  limited  performance  in  envi¬ 
ronments  with  electromagnetic  fields.  As  clinical  environments  require  large  amounts 
of  equipment,  it  is  impossible  to  control  for  electromagnetic  fields.  To  account  for  this 
high  rate  of  error,  activities  that  are  covered  by  the  current  approaches  are  macro 
movement  based  activities  such  as  entering  a  room  or  going  from  one  area  of  the  hos¬ 
pital  to  another.  Current  systems  are  limited  in  documenting  activities  that  occur  in 
smaller  area  as  the  sensors  cannot  discriminate  location  in  these  environments  with  ac¬ 
ceptable  accuracy. 

Video-based  tracking  suffers  from  similar  issues.  The  locations  of  cameras  are 
fixed.  Areas  need  to  be  analyzed  to  ensure  that  camera  cover  all  parts  that  need  to  be 
monitored.  In  addition,  real-time  analysis  of  videos  for  entity  recognition  can  suffer 
from  typical  video  processing  problems,  such  as  occlusion  of  entities  by  other  entities, 
noise,  motion  blur,  uneven  lighting  and  so  on.  This,  coupled  with  the  requirements  of 
privacy  and  security,  often  render  video-based  capture  unusable. 

As  both  qualitative  and  quantitative  methods  have  advantages  and  disadvantag¬ 
es,  an  improved  solution  for  workflow  monitoring  can  be  obtained  by  combining  the 
two  types  of  methods.  In  this  work,  workflow  monitoring  is  perfonned  using  RID  tags 
in  conjunction  with  ethnographic  observations.  Unlike  infra  red  tags,  RID  tags  do  not 
require  a  line  of  sight  with  other  tags  to  record  information.  Hence,  we  utilize  RID  tags 
to  gather  the  quantitative  data.  Observations  gathered  complement  RID  data  by  provid¬ 
ing  a  detailed  description  of  communication  and  interaction  activities  that  cannot  be 
captured  using  the  tags.  This  is  the  approach  taken  for  the  development  of  the  system 
described  in  this  paper. 
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3  Methods 

In  general,  workflow  can  be  described  by  (a)  the  underlying  cognitive  processes  that 
drive  decision  making,  (b)  physical  movement,  and  (c)  interaction  and  communication 
activities.  In  this  work,  we  present  the  framework  for  a  workflow  analysis  system  that 
combines  qualitative  and  quantitative  methods  of  data  collection  to  capture  each  of 
these  three  activities.  Figure  1  depicts  the  types  of  activities  that  can  be  captured  using 
the  qualitative  and  quantitative  data  gathered.  RID  tags  can  provide  quantitative  infor¬ 
mation  about  movement  activities,  in  addition  to  some  basic  interaction  statistics  such 
as  proximity  between  two  or  more  clinicians  and  time  spent  at  particular  locations. 
These  statistics  could  be  utilized  to  model  the  movement  patterns  of  clinicians  in  the 
environment  and  the  overall  behavior  of  the  clinical  team.  RID  tags,  however  cannot 
gather  information  about  specific  details  of  communication  between  clinicians,  or  de¬ 
tails  about  the  thought  processes  of  clinicians  that  result  in  a  particular  situation.  We 
rely  on  qualitative  observations  to  provide  this  information. 


Figure  1.  Overview  of  activities  captured  and  tools  utilized 


Traditionally  qualitative  data  collection  requires  observers  to  focus  on  both  cog¬ 
nitive  activities  and  movement  activities.  This  is  in  addition  to  collecting  detailed  in¬ 
formation  about  the  time  of  activity  initiation  and  the  sequence  of  activities  in  a 
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workflow.  Our  system  offers  the  means  to  offload  the  task  of  recognizing  movement 
based  activities  and  a  subset  of  interaction  and  communication  activities  to  the  automat¬ 
ic  algorithms  that  process  incoming  quantitative  data  from  RID  tags  and  estimate  the 
activity  being  performed  by  the  team  (as  depicted  in  figure  1).  Our  system  can  theoret¬ 
ically  capture  any  movement  activities  that  require  team  members  to  move  at  least  8 
inches17.  The  communication  and  interaction  activities  that  show  movement  patterns  of 
a  single  entity  or  a  group  of  entities  can  be  suitably  recognized  by  the  developed  sys¬ 
tem. 

An  example  of  such  an  interaction  activity  would  be  “patient  arriving”.  Typical¬ 
ly,  interactions  with  two  or  more  entities  can  be  monitored  using  RID  information. 
When  a  patient  arrives  in  a  trauma  room,  the  team  members  tend  to  converge  at  a  trau¬ 
ma  pod.  This  is  an  interaction  and  communication  activity  that  can  be  captured  by  using 
movement  as  a  proxy.  As  a  general  rule,  any  interaction  and  communication  activity 
that  is  accompanied  with  measurable  movement  can  be  captured  by  this  system  and 
recognized.  Following  the  same  logic,  any  communication  or  interaction  activity  that  is 
not  accompanied  by  movement  cannot  be  captured  by  the  automatic  analysis  system. 

In  the  future  we  aim  to  include  additional  sensors  that  can  give  us  more  detailed 
infonnation  on  some  of  the  activities.  Incorporation  of  audio  recording  would  facilitate 
automated  tracking  of  communication  between  entities.  Further,  in  order  to  increase  the 
resolution  of  the  movement  activities  that  can  be  tracked  through  our  system  we  aim  to 
use  accelerometers.  Acceleration  measurements  could  provide  information  on  whether 
the  tagged  entity  is  moving.  This  would  enable  the  refinement  of  movement  tracking 
and  activity  recognition.  The  presented  framework  accounts  for  these  future  additions 
to  allow  for  seamless  integration. 

3,1  Conceptual  Framework 

In  the  given  framework,  we  collect  two  different  streams  of  data, 

(a)  Qualitative  data  from  observers,  and, 

(b)  Quantitative  data  gathered  from  the  RID  tags. 

Both  the  qualitative  data  and  quantitative  data  are  obtained  from  standardized 
sources.  While  time-stamped  quantitative  data  is  retrieved  from  the  RID  tags,  observa- 
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tions  are  generated  by  observers  using  an  activity  tracking  software  tool  depicted  in 
Figure  2. 


Bn]  Physician  and  Nurse  Activities  in  ED 


Physician  ]  Nurse  | 

Nurse  Primary  Duties: 

I  |  Triage  patients 

I  |  Perform  assessment/reassessment  of  patients 

□  Monitor  patients  Neurological  status 

□  Monitor  patient's  Respiratory  status 

I  |  Monitor  patient  's  Hemodynamic  status 
I  |  Document  care 

□  Patient  and  family  education 

□  Administer  and  document  medication  ordered  by  physician 
I  |  Administer  blood  and  blood  product 

I  |  Receive  laboratory  results 

□  ekg 

□  NGT 

I  |  Foley  catheter  insertion 
I  |  Moderate  (procedural)  sedation 

|  |  Perform  venipunctures  including  starting  intravenous  infusions  and  drawing  blood  specimens 
I  |  Communicate  with  other  health  care  team  members 
I  |  Assist  with  Resuscitation 

□  Assist  with  Intubation 

□  Assist  with  Central  line  placement 
I  |  Assist  with  Chest  tube  insertion 

I  |  Assist  with  Orthopedic  treatments 

□  Assist  with  Radiological  studies  at  the  bedside 

□  Assist  with  Laceration  repair  (suture/staple) 

Nurse  Description: 


Submit 


Figure  2.  Observation  tracking  tool 


The  tool  contains  a  list  of  commonly  occurring  activities  for  the  Nurse  and  Phy¬ 
sician.  The  activities  chosen  were  based  on  an  ontology  developed  by  Jiajie  Zhang 
based  on  his  prior  work  on  analyzing  the  workflow  in  emergency  departments".  Ob¬ 
servers  may  select  an  appropriate  activity  from  the  list  provided  and  add  detailed  com¬ 
ments  a  description  text  box.  The  observations  are  then  automatically  dated  and  timed 
and  stored  in  the  output  observation  file.  In  this  way  time-stamped  data  is  obtained  for 
both  qualitative  and  quantitative  data  sources.  This  makes  synchronization  of  the  two 
data  streams  possible. 

Quantitative  data  is  obtained  using  active  RID  tags  to  gather  data.  Active  RID 
tags  have  an  inbuilt  power  source,  hence  the  name  active.  In  addition  to  being  portable, 
active  tags  use  low  levels  of  energy  ensuring  that  they  do  not  interfere  with  other  devic- 


Nurse  Secondary  Duties: 

[~|  Answer  phones 
I  |  Care  after  death 

□  Transport  patients  to  Radiology  Departments 

□  Transport  patients  to  Nuclear  Medicine 

I  |  Transport  patients  Communicate  with  family  members 
I  |  Transport  patients  Medical -Surgical  and  Intensive  Care  Units 

□  Data  entry  and  other  clerical  computer  task 

□  Food,  bathroom,  etc 


Nurse  Tertiary  Duties: 

□  Phone  calls  not  related  to  patient  care 

I  |  Communications  with  other  healthcare  professional  not  related  to  patient  care 
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es,  such  as  telephones  and  other  network  connections  found  in  a  health  care  setting. 
This  is  a  vital  requirement  for  the  system.  We  chose  off  the  shelf  tags  available  from 
SNiF®.  These  are  shown  in  figure  3.  The  tags  record  encounters  with  other  tags  (tag-tag 
encounter)  and  base  stations  (tag-base  encounter).  For  each  encounter  or  interaction,  the 
tags  record 

(a)  Identification  number  of  the  tag  or  base  station  detected 

(b)  Time  and  date  of  encounter,  and 

(c)  The  received  signal  strength  indication  (RSSI)  value. 


Figure  3.  Active  RID  (SNiF®)  tag  and  base  station 


The  RSSI  value  provides  proximity  information.  This  value  is  inversely  related 
to  the  distance  between  the  interacting  tags.  Consequently,  it  can  be  used  to  measure 
approximate  distances  between  tags  and  base  stations  involved  in  an  interaction.  The 
assessment  of  the  temporally  changing  distance  between  entities  (people  and  equipment 
alike),  enables  inferring  the  initiation  of  activities  such  as  a  resident  leaving  trauma,  or 
a  nurse  documenting  the  case  at  the  nurse’s  station.  Such  utilization  of  proximity  in¬ 
formation  makes  this  a  crucial  metric  for  the  functioning  of  the  activity  recognition  sys¬ 
tem. 

Base  stations  can  be  considered  as  fixed  tags  that  provide  tag  location  informa- 
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tion.  We  place  base  stations  at  critical  locations  in  the  critical  care  environment.  It  was 
found  that  in  a  Trauma  unit,  the  trauma  bays,  nurse’s  station  and  entry  and  exit  points 
were  some  of  the  key  locations.  These  locations  will  vary  from  site  to  site,  depending 
on  variations  in  the  workflow.  Placing  base  stations  in  these  locations  enables  the  deter¬ 
mination  of  activities  that  occur  in  specific  regions.  Combining  tag-tag  and  tag-base 
encounter  information  we  can  obtain  an  abstraction  of  interaction  and  communication 
activities  between  various  entities  in  the  clinical  environment. 

In  order  to  develop  our  quantitative  analysis  system  there  is  a  need  for  training 
and  validating  the  models.  The  procedures  for  data  collection  to  train  our  quantitative 
system  are  as  follows.  Firstly,  we  identify  certain  movement  activities,  communication 
activities  and  interaction  activities  that  are  of  interest.  Subsequently,  we  place  the  RID 
tags  on  entities  and  collect  data  on  the  movements  of  entities  when  performing  those 
activities.  This  data  is  then  employed  to  train  a  model  for  automatic  recognition  of 
these  activities  in  the  future  observations. 

3.1.1  An  illustrative  example 

Let  us  see  an  illustrative  example  of  how  some  activities  in  the  clinical  environments 
can  be  captured  by  appropriate  placement  of  tags  and  base  stations  (which  are  similar  to 
tags  in  functionality  but  have  a  larger  form  factor.  See  figure  3).  Consider  the  scenario 
representing  patient  arrival  is  depicted  in  Figure  4.  Firstly,  key  members  of  the  patient 
care  team  (resident,  nurse  and  so  on)  gather  by  the  bed  of  the  patient.  Following  this, 
examination  of  the  patient  takes  place.  A  resident  may  move  to  the  telephone  to  consult 
or  the  nurse  may  move  to  the  nurse’s  station  to  document  details  of  the  encounter.  All 
these  activities  are  linked  to  entities  perfonning  some  type  of  movement  in  the  environ¬ 
ment. 
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Figure  4.  Scenario:  Patient  arrival  at  a  trauma  unit 


Formally  we  can  express  this  sequence  of  activities  in  terms  of  time  as 

(i)  At  time  ti :  Patient  arrives  at  the  trauma  unit  and  is  sent  to  the  trauma  bay 

(ii)  At  time  h-  The  nurse  and  a  resident  check  in  on  the  patient 

(iii)  At  time  t3:  The  resident  seeks  a  phone  consult  while  the  nurse  heads  over  to  the 
station  to  continue  with  documentation. 

In  our  diagram  (Figure  4),  ‘P’  refers  to  the  patient;  ‘N’  refers  to  the  nurse  and 
‘R’  to  the  resident  on  call.  The  black  solid  dots  denote  location  of  base  stations 
Base  stations  were  placed  at  various  key  locations;  one  at  each  trauma  bay,  one  near  the 
phone  and  the  other  near  the  computer.  For  these  given  sequence  of  events,  the  follow¬ 
ing  are  the  trends  we  see  in  the  data  derived  from  the  tags. 

(i)  At  time  t2 :  Tags  R  and  N  get  close  to  Bi. 

(ii)  At  time  t3:  Tag  N  is  very  close  to  B5  and  Tag  R  is  very  close  to  B6. 

With  the  initial  setup  phase  we  know  that  Bi  is  trauma  bay  1,  we  can  assume 
that  the  patient  is  being  managed  by  the  nurse  and  resident  at  time  t2  and  that  the  patient 
arrived  at  the  unit  sometime  before  t2.  Therefore,  at  time  t3,  the  system  can  probabilisti- 
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cally  estimate  that  the  nurse  was  documenting  the  patient  report,  and  the  resident  was 
seeking  a  phone  consult.  While  the  scenario  presented  is  a  simplification  of  the  total 
process,  it  provides  a  conceptual  view  of  how  we  can  track  activities  through  tags.  In 
reality,  activity  models  generated  can  be  more  complex.  The  models  would  be  required 
to  handle  variations  in  activities  performed  while  classifying  them  accurately.  A  pattern 
recognition  approach  is  required  to  handle  such  variations.  The  approach  taken  in  this 
work  to  classify  activities  is  discussed  in  the  following  section. 

3.2  Activity  modeling  and  recognition  using  Hidden  Markov  Modeling 
Hidden  Markov  Modeling  (HMM)  is  a  probabilistic  modeling  tool  that  is  usually  em¬ 
ployed  for  temporal  sequence  analysis  and  have  been  effectively  used  in  movement 
analysis,  gesture  and  speech  recognition  applications.18'19  An  HMM  models  a  temporal 
sequence  of  events  (called  an  observation  sequence)  in  terms  of  a  state  machine,  in 
which  the  current  state  of  the  model  is  probabilistically  dependent  on  the  previous 
states.  A  well-trained  HMM  activity  model  can  detect  the  temporal  activities  that  the 
HMM  has  been  trained  for. 

As  with  any  method,  HMM  based  activity  recognition  has  certain  advantages 
and  disadvantages.  The  key  disadvantage  of  HMMs  lies  in  the  fact  that  the  amount  of 
data  that  is  required  to  train  an  HMM  is  very  large.  Another  issue  with  HMMs  is  that 
they  require  positive  data  to  train  with,  i.e.  in  order  to  effectively  train  an  HMM  to  rec¬ 
ognize  a  class  of  activities,  we  require  a  carefully  constructed  training  set  that  best  de¬ 
scribes  the  activity.  However,  these  disadvantages  are  outweighed  by  a  trained  HMM’s 
capability  to  handle  variations  in  the  final  style  of  execution  of  an  activity.  Activities 
may  be  performed  in  a  different  manner  in  critical  care  environments  and  it  is  important 
that  the  model  of  activities  accounts  for  these  variations.  By  training  the  HMM  system 
in  a  robust  manner,  it  is  possible  to  recognize  the  motion  and  some  communication  ac¬ 
tivities  regardless  of  the  deviations  for  our  application.  In  addition,  HMMs  scale  well  as 
they  can  be  trained  to  learn  activities  incrementally.  New  activities  can  be  trained  for 
without  affecting  models  of  previously  learned  activities.  For  these  reasons,  we  chose 
HMMs  for  the  development  of  activity  models  and  activity  recognition. 

Activity  recognition  using  HMMs  is  a  2  step  process.  It  involves  (i)  training 
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HMMs  for  specific  activity  models  and  (ii)  testing  the  HMMs  for  their  recognition  ac¬ 
curacy  with  annotated  test  samples.  In  order  to  develop  robust  activity  HMMs,  we  first 
require  data  that  describes  the  activity.  This  data  is  obtained  from  the  RID  tags.  More 
specifically,  the  data  utilized  is  the  RSSI  value  of  each  tag-base  encounter  gathered  dur¬ 
ing  data  collection.  We  collect  this  data  for  the  activities  of  interest  in  multiple  samples. 
We  utilize  half  of  the  samples  for  training  the  HMMs  and  retain  the  rest  for  testing  the 
developed  models.  A  database  of  samples  for  each  activity  facilitates  training  the 
HMMs  for  each  activity,  thereby  creating  a  library  of  HMM  activity  models  for  each 
activity.  The  training  of  HMM  activity  models  is  achieved  using  the  Baum- Welch  algo¬ 
rithm20. 

Once  a  library  of  HMMs  is  built  with  one  HMM  for  each  activity,  the  developed 
models  can  be  tested.  The  testing  of  an  activity  sample  proceeds  by  firstly,  estimating 
the  probability  that  the  sample  movement  belongs  to  the  library.  This  is  achieved  using 
the  Forward-Backward  procedure  for  each  of  the  HMM’s  in  the  library.  The  HMM 
that  yields  the  highest  probability  for  the  test  sequence  is  determined  to  be  the  type  of 
activity  that  the  movement  sequence  belongs  to.  The  accuracy  of  recognition  is  meas¬ 
ured  as  the  ratio  of  the  number  of  correctly  identified  test  sequences  to  the  total  number 
of  test  sequences.  In  this  manner,  activity  models  are  developed  and  tested  for  activity 
recognition.  The  recognized  activities  can  then  be  visualized  in  3D  virtual  environ¬ 
ments. 

3.3  Visualization  of  workflow  in  a  virtual  environment 

The  automated  system  for  workflow  analysis  generates  a  series  of  activities  that  takes 
place  in  the  clinical  setting.  Visualizing  workflow  in  3D  enables  researchers  and  clini¬ 
cians  alike  to  easily  grasp  the  activities  that  make  up  the  workflow.  In  addition  to  ena¬ 
ble  researchers  review  workflow  in  a  novel  way,  the  configurable  VR  visualizations  can 
also  be  employed  for  educational  purposes.  For  example,  a  resident  would  be  able  to  go 
experience  a  trauma  from  the  perspective  of  the  attending  or  nurse.  This  kind  of  confi¬ 
gurability  would  enable  the  cross-training  of  clinical  teams.  The  visualizations  can  also 
be  used  to  educate  clinicians  by  illustrating  cases  of  optimal  workflow  in  relation  to 
error-prone  workflow. 
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In  the  domain  of  healthcare,  virtual  reality  has  been  used  to  develop  simulations 
for  training  of  cognitive  and  psychomotor  surgical  skills  and  clinical  decision  making 
skills.  '  However,  there  is  a  lack  of  VR-based  solutions  for  visualization  of 
workflows  and  error  scenarios  even  though  such  systems  may  have  a  major  role  to  play 
in  error  prevention  and  mitigation.  We  can  employ  online  VR  environments  such  as 
Second  Life®  (http://secondlife.com/)  and  Active  Worlds® 
(http://www.activeworlds.com/)  for  such  visualizations.  In  this  stage  of  the  work,  we 
have  developed  a  standalone  system  that  could  be  employed  for  such  visualizations 
employing  an  open  source  gaming  engine  called  Irrlicht  (www.irrlicht.net). 

A  sample  virtual  trauma  unit  (see  figure  5)  was  developed  to  mimic  the  trauma 
unit  at  Banner  Good  Samaritan  Medical  Center  which  is  the  site  of  development  for  the 
project.  The  virtual  trauma  room  consists  of  four  trauma  pods  or  beds.  The  nurses’  sta¬ 
tion  faces  the  trauma  pods.  A  computer  and  phone  are  key  components  that  are  included 
in  the  design  of  the  nurses’  station.  Two  exit  doors  are  present  in  either  side  of  the 
trauma  room.  These  details  are  synchronous  with  the  test  and  real-world  set  up.  The 
current  simulation  contains  3  basic  characters  -  the  patient,  resident  and  the  nurse.  The 
number  and  type  of  models  to  be  utilized  depend  on  the  entities  studied  in  the  real- 
world.  Models  of  the  characters  are  built  using  modeling  software  (Maya  and  3dMax; 
http://usa.autodesk.com/).  Once  the  models  are  developed  they  can  be  controlled  in  the 
simulated  world  programmatically. 


Figure  5.  Virtual  trauma  unit  for  workflow  visualization 
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In  order  to  obtain  VR  simulations  of  the  workflow,  the  system  generates  a  list  of 
activities  making  up  the  workflow.  These  activities  are  then  manually  fed  into  the  vi¬ 
sualization  engine  to  create  the  simulations.  Currently,  this  stage  of  visualization 
process  is  completed  offline.  VR  simulations  created  in  this  manner  present  a  simulated 
view  of  real-world  events.  This  is  valuable  to  clinicians  and  researchers  in  highlighting 
the  main  events  in  the  workflow  within  the  context  of  the  clinical  environment. 

4  System  Evaluation 

The  hypothesis  being  evaluated  is  that  clinical  activities  involving  movement  patterns 
can  be  recognized  by  the  HMM  based  activity  recognition  system.  In  addition,  the 
evaluation  seeks  to  quantify  the  accuracy  of  activity  recognition  (the  ratio  of  the  num¬ 
ber  of  correctly  identified  activities  to  the  total  number  of  test  activities).  All  observa¬ 
tions  and  data  gathering  were  perfonned  after  obtaining  approval  from  Institutional  Re¬ 
view  Boards  of  involved  institutions. 

4.1  System  evaluation  setup 

In  order  to  test  the  HMM  based  activity  recognition  system,  we  simulated  15  Trauma 
activities  (listed  in  Table  1)  in  a  lab  setting,  (depicted  in  Figure  6)  with  10  tags  and  6 
base  stations.  These  activities  were  simulations  of  clinical  activities.  In  order  to  simu¬ 
late  potential  activities  in  a  lab  setting  we  observed  commonly  occurring  movement 
tasks  in  the  Trauma  unit,  an  example  being  “physician  moving  to  phone  for  a  consult” 
(Activity  A 13).  Figure  6  depicts  the  lab  setup  for  testing  and  Table  1  summarizes  the 
movement  patterns  and  clinical  descriptions  of  the  15  activities. 


19 


Bed  1 


Bed  2 


f - 

Id 

NURSE  STATION 

89 

e 

o 

Figure  6.  Test  setup  for  simulated  clinical  activities 


Activity 

Movement 

Clinical  Description 

A1 

l-to-2 

Paged  physician/nurse  tends  to  patient  on  bed  1 

A2 

2-to-3 

Physician/Nurse  moves  to  treat  patient  on  bed  2 

A3 

3-to-4 

Physician/Nurse  leaves  Trauma  through  entry/exit  1  after  visiting  patient 

on  bed  2 

A4 

4-to-5 

Physician/Nurse  enters  Trauma  through  entry/exit  1  and  attends  to  the 

phone 

A5 

5-to-6 

Physician/Nurse  after  attending  to  a  phone  call  move  to  use  the  computer 

at  the  nurse  station 

A6 

6-to- 1 

Physician/Nurse  leaves  Trauma  through  entry/exit  2 

A7 

1  -to-4 

Physician/Nurse  enter  and  leave  Trauma 

A8 

4-to-6 

Physician/Nurse  enter  Trauma  through  entry/exit  1  and  move  to  use  the 

computer  at  the  nurse  station 

A9 

6-to-2 

After  using  the  computer  physician/nurse  move  to  treat  patient  on  bed  1 

A10 

2-to-4 

After  visiting  patient  on  bed  1,  physician/nurse  leaves  Trauma  through  en¬ 
try/exit  1 
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All 

5-to-l 

After  attending  a  phone  call,  physician/nurse  leaves  Trauma  through  en¬ 
try/exit  2 

A12 

1  -to-3 

Paged  physician/nurse  attends  to  patient  on  bed  2 

A13 

3-to-5 

After  visiting  patient  on  bed  2  physician  seeks  a  phone  consult 

A14 

5-to-2 

After  completing  a  phone  call  physician/nurse  moves  to  treat  patient  on 

bed  1 

A15 

3-to-6 

After  treating  patient  on  bed  2  physician/nurse  move  to  use  the  computer 

at  the  nurse  station 

Table  1 .  Activity  list  and  corresponding  clinical  descriptions 


The  setup  for  the  testing  involved  the  creation  of  a  20ft  by  20  ft  grid  in  a  lab  set¬ 
ting  (grid  lines  depicted  in  Figure  6).  Six  base  stations  (depicted  by  black  solid  circles) 
we  placed  in  predefined  locations  (Base  1  and  4  at  Entry/Exit  points  2  and  1  respective¬ 
ly;  Bases  2  and  3  at  Beds  1  and  2;  Base  5  at  the  phone  on  nurse  station;  Base  6  at  the 
computer  on  the  nurse  station).  This  is  congruous  with  base  station  setup  in  the  real- 
world  scenario. 

We  gathered  movement  data  for  the  15  sample  activities  listed  in  Table  1.  For 
each  RID  tag-base  pair  or  tag-tag  pair  an  encounter  is  recorded  every  3  to  4.5  seconds. 
This  data  is  captured  in  a  time  modulated  manner  i.e.  encounter  information  is  commu¬ 
nicated  by  detecting  differences  in  the  time  of  the  encounter  rather  than  the  frequency. 
This  results  in  a  sparse  matrix  when  considering  the  entire  tag-base  station  configura¬ 
tion.  Figure  7  depicts  a  sample  of  the  matrix  generated.  The  encounters  of  a  tag  X  with 
base  stations  A,  B  and  C  (gray  filled  boxes)  are  shown  in  a  60  second  long  timeline.  We 
use  linear  interpolation  to  fill  missing  data  in  this  sparse  matrix.  While  this  methodolo¬ 
gy  provides  an  RSSI  value  for  all  base  stations  at  all  instances,  it  adds  some  noise  to 
our  system  that  may  affect  the  overall  activity  recognition  accuracy. 
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Figure  7.  Sparse  matrix  of  tag-base  encounters  (gray  fill  indicating  an  encounter  record 

with  some  tag) 


For  each  of  these  activities,  we  gathered  10  samples  of  data.  Each  sample  in¬ 
volved  a  tagged  entity  (researcher)  following  the  movement  pattern  prescribed  for  the 
activity.  Each  sample  performed  with  10  different  tags,  totaling  100  samples  for  each 
activity.  This  ensured  sufficient  randomization  of  activity  movements,  accounting  for 
inter-tag  variability  as  well.  A  total  of  1500  samples  (15  activities  x  10  samples  x  10 
tags)  were  gathered  for  testing. 

Out  of  the  100  samples  gathered  for  each  activity,  50  samples  were  used  to  train 
the  HMM  for  activity  recognition,  and  the  other  50  were  used  as  a  testing  set  to  eva¬ 
luate  the  algorithms’  accuracy. 


Activity 


Figure  8.  Hidden  Markov  Model  (HMM)  based  activity  recognition  results 
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4.2  Results  of  HMM  training 

Figure  8  summarizes  the  recognition  accuracy  for  the  15  motion  patterns  (A1  -  A 15) 
elucidated  in  Table  1 .  Recognition  accuracy  is  the  ratio  of  the  number  of  activities  cor¬ 
rectly  identified  to  the  total  number  of  activities  used  for  testing.  A  mean  recognition 
accuracy  of  87.5%  was  obtained,  with  a  maximum  of  90.5%  and  minimum  of  84.5%. 
The  analysis  of  the  incorrectly  classified  test  samples  revealed  that  misclassifications 
were  a  result  of  variations  in  the  training  set.  As  discussed  previously,  HMMs  require  to 
be  trained  on  a  well  controlled  sample  that  best  represent  the  activity.  Obtaining  train¬ 
ing  data  from  real-world  scenarios  are  abound  to  have  variations  that  may  compromise 
the  quality  of  models  generated.  This  is  a  limitation  of  utilizing  HMM  models  with 
RSSI  values  alone  for  activity  recognition.  Additional  sensors  such  as  accelerometers 
could  be  utilized  in  conjunction  with  RID  tags  to  improve  the  activity  recognition  rates. 
This  is  a  part  of  our  future  work. 

5  Discussion 

We  are  currently  developing  models  for  various  activities  in  the  workflow  in  a  Trauma 
unit.  The  primary  challenges  to  training  HMMs  for  various  activities  lie  in  (i)  develop¬ 
ing  a  controlled  set  of  samples  that  best  represent  the  activity  being  modeled  and  (ii)  the 
current  limitations  of  RID  tags.  The  linear  interpolation  adopted  for  dealing  with  miss¬ 
ing  data  introduced  further  errors  into  the  system.  Our  future  work  includes  improving 
the  recognition  accuracy  of  the  system  by  (i)  increasing  the  sampling  frequency  of  tags 
(ii)  using  alternate  methods  of  interpolation  to  fill  the  sparse  matrix  and  (iii)  incorporat¬ 
ing  accelerometers  with  existing  tags  to  refine  data  describing  the  movement.  In  addi¬ 
tion,  as  the  evaluation  of  the  system  was  conducted  in  a  controlled  environment  with  a 
limited  number  of  tags,  further  evaluation  and  testing  with  multiple  tags  in  critical  care 
units  would  be  required  to  complete  the  validation  of  this  system. 

Initial  data  on  the  usability  of  the  current  application  gathered  during  informal 
testing  revealed  positive  usability  and  utility  results.  The  experts  pointed  to  the  ability 
of  VR  simulations  to  focus  on  process  and  how  this  could  be  employed  for  several  ap¬ 
plications  in  the  healthcare  system.  This  is  a  major  contribution  of  this  work  as  to  date 
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there  has  been  a  lack  of  virtual  worlds  that  are  driven  by  real  world  data  in  the  domain 
of  healthcare. 

6  Conclusion  and  Future  Work 

The  research  and  system  described  in  this  work  lays  the  foundation  for  development  of 
virtual  worlds  that  are  driven  by  real  world  data  and  real  world  needs.  This  would  pro¬ 
vide  cognitive  science  researchers  another  dimension  of  data  for  clinical  workflow 
analysis.  Similar  to  a  black  box,  the  system  provides  a  means  to  track  all  entities  and  a 
subset  of  their  activities  in  the  environment.  In  addition,  the  methods  described  can  be 
applied  to  assess  workflow  not  only  in  Trauma,  but  to  any  environment  in  which  mul¬ 
tiple  team  members  are  sufficiently  mobile  and  geographically  dispersed  such  that  their 
movements  might  meaningfully  reflect  their  activities. 

Currently  the  system  is  capable  of  augmenting  conventional  data  collection  me¬ 
chanisms  to  offer  multidimensional  activity  information  that  allows  observers  to  focus 
on  cognitive  details  rather  than  simply  annotating  movement  activities.  Another  use  for 
this  system  is  for  the  retrospective  analysis  of  data.  While  existing  workflow  analysis 
systems  support  this  type  of  analysis,  the  system  described  in  the  work  can  provide  ad¬ 
ditional  information  that  can  be  used  to  direct  researchers  to  key  events  that  need  to  be 
analyzed  further.  Visualization  of  workflow  is  another  benefit  of  using  this  tool.  In  ad¬ 
dition  to  enhancing  our  understanding  of  clinical  workflow,  such  visualizations  can  also 
be  used  as  an  educational  tool  for  training  residents  and  nurse  trainees  in  identifying 
errors  and  preventing  the  potential  errors  from  leading  to  adverse  events. 

Future  work  will  include  developing  more  complex  activity  models  and  im¬ 
proving  the  HMM  recognition  rates.  Additional  data  gathering  for  HMM  modeling  of 
activities  and  inclusion  of  audio  recordings  and  accelerometers  will  enable  the  enrich¬ 
ment  of  information  gathered.  This  will  provide  more  information  about  the  workflow 
and  improve  the  capability  of  the  system  to  capture  and  analyze  workflow  automatical¬ 
ly. 


24 


References 

1.  Leape  L,  Berwick  D.  Five  years  after  to  err  is  human:  what  have  we  learned? 
Jama.  1995;293:2385-90. 

2.  Brixey  JJ,  Robinson  DJ,  Turley  JP,  Zhang  J.  The  roles  of  MDs  and  RNs  as  in¬ 
itiators  and  recipients  of  interruptions  in  workflow.  International  Journal  of  Medical 
Informatics.  2008 ;In  Press,  Corrected  Proof. 

3.  Malhotra  S,  Jordan  D,  Shortliffe  E,  Patel  VL.  Workflow  modeling  in  critical 
care:  Piecing  together  your  own  puzzle.  Journal  of  Biomedical  Informatics. 
2007;40(2):81-92. 

4.  Goldspink  C.  Modelling  social  systems  as  complex:  Towards  A  social  simula¬ 
tion  meta-model.  Journal  of  Artificial  Societies  and  Social  Simulation. 
2000;3(2):  [Available  from:  http://www.soc.surrev.ac.Uk/JASSS/3/2/l.htmn. 

5.  Waldrop  WM.  Complexity:  The  Emerging  Science  at  the  Edge  of  Order  and 
Chaos;  1992. 

6.  Holland  J.  Hidden  Order:  how  adaptation  builds  complexity.  NY:  Addison- 
Wesley;  1995. 

7.  Horsky  J,  Gutnik  L,  Patel  VL.  Technology  for  emergency  care:  cognitive  and 
workflow  considerations.  AMIA  Annual  Fall  Symposium;  2006;  2006.  p.  344-8. 

8.  Laxmisan  A,  Hakimzada  F,  Sayan  OR,  Green  RA,  Zhang  J,  Patel  VL.  The  mul¬ 
titasking  clinician:  Decision-making  and  cognitive  demand  during  and  after  team  han- 
doffs  in  emergency  care.  International  Journal  of  Medical  Informatics.  2007;76(11- 
12):80 1- 1 1 . 

9.  Alwan  M,  Dalai  S,  Mack  D,  Kell  SW,  Turner  B,  Leachtenauer  J,  et  al.  Impact  of 
Monitoring  Technology  in  Assisted  Living:  Outcome  Pilot.  IEEE  Transactions  On  In¬ 
formation  Technology  in  Biomedicine.  2006;  10(1):  192-8. 

10.  Ostbye  T,  Lobach  DF,  Cheesborough  D,  Lee  AMM,  Krause  KM,  Hasselblad  V, 
et  al.  Evaluation  of  an  Infrared/Radiofrequency  Equipment-Tracking  System  in  a  Ter¬ 
tiary  Care  Hospital.  Journal  of  Medical  Systems.  2003;27(4):367-80. 

1 1 .  Stanford  V.  Using  Pervasive  Computing  to  Deliver  Elder  Care.  IEEE  Pervasive 
Computing.  2002;  1(1 ):  1 536-268. 

12.  Matthai  P,  Kenneth  PF,  Mike  P,  Donald  JP,  Dieter  F,  Henry  K,  et  al.  Inferring 
Activities  from  Interactions  with  Objects.  IEEE  Pervasive  Computing.  2004;3(4):50-7. 

13.  Schrooyen  F,  Baert  I,  Truijen  S,  Pieters  L,  Denis  T,  Williame  K,  et  al.  Real  Time 
Location  System  over  WiFi  in  a  Healthcare  Environment.  The  Journal  on  Information 
Technology  in  Healthcare.  2006;4(6):401-16. 

14.  Chen  C,  Liu  C,  Li  Y,  Chao  C,  Liu  C,  Chen  C,  et  al.  Pervasive  Observation  Med¬ 
icine:  The  Application  of  RFID  to  Improve  Patient  Safety  in  Observation  Unit  of  Hos¬ 
pital  Emergency  Department:  IOS  Press;  2005. 


25 


15.  Fry  EA,  Lenert  LA.  MASCAL:  RFID  tracking  of  patients,  staff  and  equipment 
to  enhance  hospital  response  to  mass  casualty  events.  AMIA  Annual  Symposium  Pro¬ 
ceedings.  2005:261-5. 

16.  Hauptmann  AG,  Gao  J,  Yan  R,  Qi  Y,  Yang  J,  Wactlar  HD.  Automated  analysis  of 
nursing  home  observations.  Pervasive  Computing,  IEEE.  2004;3(2):  15-21. 

17.  Islam  GS.  An  Indoor  Location  Finding  and  Fall  Detection  System  Using  Sensor 
Integration  with  RFID  Network  and  Accelerometer  Data.  Tempe:  Arizona  State  Univer¬ 
sity;  2009. 

18.  Kahol  K,  Smith  M,  Ferrara  J,  Tripathi  P,  Leibowitz  A,  Panchanathan  S.  Gesture 
Based  Hand  Movement  Analysis  and  Haptic  Feedback  for  Surgical  Training  [selected 
as  best  poster].  Poster  presentation  at  the  14th  Meeting  of  Medicine  Meets  Virtual  Rea- 
lityMedicine  Meets  Virtual  Reality  14.  2006. 

19.  Rabiner  LR.  A  tutorial  on  hidden  Markov  models  and  selected  applications  in 
speech  recognition.  Proceedings  of  the  IEEE  [0018-9219].  1989;77(2):257-86. 

20.  Dugad  R,  Desai  U.  A  tutorial  on  hidden  Markov  models.  Technical  report 
SPANN-961  Signal  Processing  and  Artificial  Neural  Networks  Laboratory,  Bombay 
(India).  1996. 

21.  Seymour  NE,  Gallagher  AG,  Roman  SA,  O’Brien  MK,  Bansal  VK,  Andersen 
DK,  et  al.  Virtual  Reality  Training  Improves  Operating  Room  Performance  -  Results  of 
a  Randomized,  Double-Blinded  Study.  Ann  Surg.  2002;236(4):458-64. 

22.  Kahol  K,  Vankipuram  M,  Smith  ML.  Cognitive  simulators  for  medical  educa¬ 
tion  and  training.  Journal  of  Biomedical  Informatics.  2009;In  Press,  Corrected  Proof. 

23.  Ahlberg  G,  Heikkinen  T,  Iselius  L,  Leijonmarck  CE,  Rutqvist  J,  Arvidsson  D. 
Does  training  in  a  virtual  reality  simulator  improve  surgical  perfonnance?  Surgical  En¬ 
doscopy.  2002;  16(1):  126-9. 


26 


